2021-01-15T18:49:55.117 INFO:root:teuthology version: 1.0.0-04fddc54 2021-01-15T18:49:55.120 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi --archive /home/teuthworker/archive/teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi/5788674 --verbose --owner scheduled_teuthology@teuthology --lock --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} --block -- /tmp/teuthology-worker.c35_0po0.tmp 2021-01-15T18:49:55.166 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-15T18:49:55.258 INFO:teuthology.run:Config: arch: x86_64 archive_path: /home/teuthworker/archive/teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi/5788674 branch: master description: rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} email: ceph-qa@ceph.io first_in_suite: false job_id: '5788674' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: centos os_version: '8.2' overrides: admin_socket: branch: master ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 osd_async_recovery_min_cost: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 filestore odsync write: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd max backfills: 2 osd max markdown count: 1000 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 0.5 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 892c51dd3c5f7108e766bea30cd5e0d801b0abd3 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 892c51dd3c5f7108e766bea30cd5e0d801b0abd3 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: master sha1: 892c51dd3c5f7108e766bea30cd5e0d801b0abd3 owner: scheduled_teuthology@teuthology priority: 80 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 892c51dd3c5f7108e766bea30cd5e0d801b0abd3 sleep_before_teardown: 0 suite: rados suite_branch: master suite_path: /home/teuthworker/src/git.ceph.com_ceph_master/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: 892c51dd3c5f7108e766bea30cd5e0d801b0abd3 tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 2 m: 1 name: isaprofile plugin: isa technique: reed_sol_van min_size: 2 objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 write_append_excl: false teuthology_branch: master timestamp: 2021-01-15_07:01:02 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.1447 2021-01-15T18:49:55.258 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_master/qa; will attempt to use it 2021-01-15T18:49:55.259 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_master/qa/tasks 2021-01-15T18:49:55.259 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-01-15T18:49:55.262 INFO:teuthology.task.internal:Checking packages... 2021-01-15T18:49:55.284 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash '892c51dd3c5f7108e766bea30cd5e0d801b0abd3' 2021-01-15T18:49:55.285 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-01-15T18:49:55.285 INFO:teuthology.packaging:ref: None 2021-01-15T18:49:55.286 INFO:teuthology.packaging:tag: None 2021-01-15T18:49:55.286 INFO:teuthology.packaging:branch: master 2021-01-15T18:49:55.286 INFO:teuthology.packaging:sha1: 892c51dd3c5f7108e766bea30cd5e0d801b0abd3 2021-01-15T18:49:55.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=master 2021-01-15T18:49:55.914 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-9044.gb9095837 2021-01-15T18:49:55.915 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-01-15T18:49:55.924 INFO:teuthology.task.internal:no buildpackages task found 2021-01-15T18:49:55.924 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2021-01-15T18:49:56.066 INFO:teuthology.task.internal.lock_machines:Locking machines... 2021-01-15T18:49:56.067 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-15T18:49:56.120 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:50:06.135 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:50:16.150 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:50:26.164 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:50:36.216 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:50:46.229 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:50:56.252 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2021-01-15T18:51:06.265 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi/5788674', 'arch': 'x86_64'} 2021-01-15T18:51:06.286 DEBUG:teuthology.lock.ops:locked ubuntu@smithi001.front.sepia.ceph.com, ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T18:51:06.333 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f05749e08c8>, signals=[15]) 2021-01-15T18:51:06.544 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of centos 8.2 2021-01-15T18:51:06.547 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 8.2 2021-01-15T18:51:06.923 INFO:teuthology.orchestra.console:Power off smithi001 2021-01-15T18:51:06.924 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-15T18:51:06.946 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:06.960 INFO:teuthology.orchestra.console:Power off smithi136 2021-01-15T18:51:06.960 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-15T18:51:06.979 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:10.970 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:11.003 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:15.082 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:15.112 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:15.293 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2021-01-15T18:51:15.394 INFO:teuthology.orchestra.console:Power on smithi001 2021-01-15T18:51:15.395 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-15T18:51:15.414 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:19.222 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:19.432 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-01-15T18:51:19.435 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:19.533 INFO:teuthology.orchestra.console:Power on smithi136 2021-01-15T18:51:19.534 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-15T18:51:19.554 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:23.546 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:23.575 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-15T18:51:23.756 INFO:teuthology.orchestra.console:Power on for smithi001 completed 2021-01-15T18:51:23.783 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2021-01-15T18:51:23.858 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish 2021-01-15T18:51:23.885 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2021-01-15T18:53:39.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:53:55.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:04.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:13.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:19.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:22.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:28.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:31.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:37.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:46.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:54:55.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:55:37.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:55:37.832 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-15T18:55:37.834 DEBUG:teuthology.orchestra.run.smithi001:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-15T18:55:38.172 INFO:teuthology.provision.fog.smithi001:Node is ready 2021-01-15T18:55:38.172 DEBUG:teuthology.orchestra.run.smithi001:> hostname 2021-01-15T18:55:38.187 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com 2021-01-15T18:55:38.188 DEBUG:teuthology.orchestra.run.smithi001:> grep smithi001.front.sepia.ceph.com /etc/hosts 2021-01-15T18:55:38.243 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001 2021-01-15T18:55:38.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname smithi001 2021-01-15T18:55:38.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i -e 's/smithi001.front.sepia.ceph.com/smithi001/g' /etc/hosts 2021-01-15T18:55:38.382 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i -e 's/172.21.15.1/172.21.15.1/g' /etc/hosts 2021-01-15T18:55:38.449 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i -e 's/smithi001.front.sepia.ceph.com/smithi001/g' /etc/hostname 2021-01-15T18:55:38.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname smithi001 2021-01-15T18:55:38.579 INFO:teuthology.provision.fog.smithi001:Deploy complete! 2021-01-15T18:55:38.579 DEBUG:teuthology.parallel:result is None 2021-01-15T18:56:01.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:56:02.041 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-15T18:56:02.041 DEBUG:teuthology.orchestra.run.smithi136:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-15T18:56:02.312 INFO:teuthology.provision.fog.smithi136:Node is ready 2021-01-15T18:56:02.313 DEBUG:teuthology.orchestra.run.smithi136:> hostname 2021-01-15T18:56:02.326 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2021-01-15T18:56:02.327 DEBUG:teuthology.orchestra.run.smithi136:> grep smithi136.front.sepia.ceph.com /etc/hosts 2021-01-15T18:56:02.380 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2021-01-15T18:56:02.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2021-01-15T18:56:02.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi136.front.sepia.ceph.com/smithi136/g' /etc/hosts 2021-01-15T18:56:02.500 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/172.21.15.136/172.21.15.136/g' /etc/hosts 2021-01-15T18:56:02.564 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi136.front.sepia.ceph.com/smithi136/g' /etc/hostname 2021-01-15T18:56:02.628 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2021-01-15T18:56:02.691 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2021-01-15T18:56:02.691 DEBUG:teuthology.parallel:result is None 2021-01-15T18:56:02.840 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com 2021-01-15T18:56:02.841 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-15T18:56:02.856 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2021-01-15T18:56:02.857 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-15T18:56:02.871 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server 2021-01-15T18:56:02.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T18:56:02.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:56:02.942 DEBUG:teuthology.orchestra.run.smithi001:> true 2021-01-15T18:56:03.098 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2021-01-15T18:56:03.154 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2021-01-15T18:56:03.154 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2021-01-15T18:56:03.209 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Linux" 2021-01-15T18:56:03.209 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8 (Core)" 2021-01-15T18:56:03.209 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2021-01-15T18:56:03.210 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2021-01-15T18:56:03.210 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8" 2021-01-15T18:56:03.210 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8" 2021-01-15T18:56:03.210 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-01-15T18:56:03.211 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2021-01-15T18:56:03.211 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-01-15T18:56:03.211 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.centos.org/" 2021-01-15T18:56:03.211 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-15T18:56:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:56:03.212 INFO:teuthology.orchestra.run.smithi001.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-01-15T18:56:03.212 INFO:teuthology.orchestra.run.smithi001.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-01-15T18:56:03.212 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-15T18:56:03.212 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-01-15T18:56:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:56:03.214 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2021-01-15T18:56:03.227 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2021-01-15T18:56:03.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T18:56:03.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:56:03.293 DEBUG:teuthology.orchestra.run.smithi136:> true 2021-01-15T18:56:03.440 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2021-01-15T18:56:03.493 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2021-01-15T18:56:03.494 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2021-01-15T18:56:03.546 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Linux" 2021-01-15T18:56:03.547 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8 (Core)" 2021-01-15T18:56:03.547 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2021-01-15T18:56:03.547 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2021-01-15T18:56:03.547 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8" 2021-01-15T18:56:03.548 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2021-01-15T18:56:03.548 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-01-15T18:56:03.548 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2021-01-15T18:56:03.548 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-01-15T18:56:03.549 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.centos.org/" 2021-01-15T18:56:03.549 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-15T18:56:03.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:56:03.549 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-01-15T18:56:03.549 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-01-15T18:56:03.550 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-15T18:56:03.550 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-01-15T18:56:03.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:56:03.551 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-01-15T18:56:03.566 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2021-01-15T18:56:03.598 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj0drSpCVPEDKuZjdqvblKC6kS3T6Cb0x4UKpCtrtU0lPM8wbPCSZ4b6yqoovZZYVp3wKDfi2zpzw/6IZaxnee+yr3KKnQkYMQVx4WcRDncTFIICku+y1wLXZQvojY+GfkNzUeAP0pdXyZQ8lA/zRy79XmO2iWYNmegg7wQAvgYviXIi0yFKxAPJ8ELapU7Jd+qbAiFnCLl7W4kFFAziRcTmahy2Y7zEJfw4YVnr05Oyjkb6WvgIcNWmJpzzzaSTinyyT+/jvI1z1pMfUB0u++i7q4C5ei3wqsQpV7oKJw3huO93WEv766BUwGdGuvf2yvzJeaecsQ0ohI1E4Nl7G+dxnhxQCun3rH+WGMqs3FLB/bXav3VCgJDbj9liqGn5VWeZHfgnq4LrINzBDnDrFikdMMpufnyOKw9eKP2gBLr3J3M9KvneuZVhnBQhp8pa0SWj1DK1Mtz98pcUmH1ErLh6CQ5/rr28CLM3u9cwdlREsgngCbshLC+gyTfs4ViJs= smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZm+NC4WpSzrSM/d7OPPqqlQcDsC6R38soVwkBKlU5x6fm8Ryk9OGISm6xL0dchgW2FMQeKlwsWODPVfQwOzQD+vZ22anx2ZXB3lVtaIFhxYAHVik65M0IrNJfA6DW7YbW/2J0EOOw9Cm649WmZ14H0fqHo4KX3ltUCClb8dT7ELJKpFdgpsoEiFI2XQxeXETWT7h0Wq18gpGxvyWxnerOWzCJ9sPfqEgw8j7SbbefAk/yrWk20FcMbfNdHQPGKWYW13/xCm95sM1WcoFzDXyjXh5/tfjSEMNzpl0+Oz3Rfrc11DM+tbKtQU+9Yw/v5kRcHil3lw80SRp4VX3Q25WgpuNfLuUOEuS/jAeOMwEUc7ZC5QYzzIBzFcnIjVvxdFKrj+VPPWcEDW1cCuivDt5n+ns7Pt+sNhT8fvhRRziVvbAimsGY71xxjKYYIhR2XUpyXHkIC63XVzrA/cPz794muF7js2igC0WjwjaddHDTWsqaIcumLSXZY1U73stdMK8= 2021-01-15T18:56:03.599 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-15T18:56:03.653 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-01-15T18:56:03.659 INFO:teuthology.task.internal:Saving configuration 2021-01-15T18:56:03.678 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-01-15T18:56:03.802 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-01-15T18:56:03.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-01-15 18:51:06.273681', 'description': '/home/teuthworker/archive/teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi/5788674', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOcEZzsDUcN3cWtHsflIBpTrMNd8Jk4mJ00E9C7tVlCXx7IZxzaR2mOWfPKfQ5XxjoB+QgZ5r3fKKKUupm3dWXc=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi001.front.sepia.ceph.com'} 2021-01-15T18:56:03.836 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-01-15 18:51:06.273702', 'description': '/home/teuthworker/archive/teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi/5788674', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCjN+r7hia/5blJqAHpmUXxLI+sLJNcaCAGIPsdEnRfs52Kg8N6+Dt8EbXoyRdd/GpTkuV1colS8SsSKWlAAc+0=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi136.front.sepia.ceph.com'} 2021-01-15T18:56:03.836 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-01-15T18:56:03.938 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-01-15T18:56:03.939 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-01-15T18:56:03.939 INFO:teuthology.run_tasks:Running task console_log... 2021-01-15T18:56:04.040 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f057842fa60>, signals=[15]) 2021-01-15T18:56:04.041 INFO:teuthology.run_tasks:Running task internal.connect... 2021-01-15T18:56:04.054 INFO:teuthology.task.internal:Opening connections... 2021-01-15T18:56:04.055 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T18:56:04.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:56:04.139 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T18:56:04.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:56:04.211 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-01-15T18:56:04.222 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2021-01-15T18:56:04.252 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2021-01-15T18:56:04.252 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2021-01-15T18:56:04.307 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Linux" 2021-01-15T18:56:04.307 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8 (Core)" 2021-01-15T18:56:04.307 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2021-01-15T18:56:04.308 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2021-01-15T18:56:04.308 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8" 2021-01-15T18:56:04.308 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8" 2021-01-15T18:56:04.308 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-01-15T18:56:04.308 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2021-01-15T18:56:04.309 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-01-15T18:56:04.309 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.centos.org/" 2021-01-15T18:56:04.309 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-15T18:56:04.309 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:56:04.309 INFO:teuthology.orchestra.run.smithi001.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-01-15T18:56:04.310 INFO:teuthology.orchestra.run.smithi001.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-01-15T18:56:04.310 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-15T18:56:04.310 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-01-15T18:56:04.310 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:56:04.311 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2021-01-15T18:56:04.326 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2021-01-15T18:56:04.341 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2021-01-15T18:56:04.342 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2021-01-15T18:56:04.395 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Linux" 2021-01-15T18:56:04.396 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8 (Core)" 2021-01-15T18:56:04.396 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2021-01-15T18:56:04.397 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2021-01-15T18:56:04.397 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8" 2021-01-15T18:56:04.398 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2021-01-15T18:56:04.398 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-01-15T18:56:04.398 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2021-01-15T18:56:04.398 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-01-15T18:56:04.399 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.centos.org/" 2021-01-15T18:56:04.399 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-15T18:56:04.399 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:56:04.399 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-01-15T18:56:04.400 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-01-15T18:56:04.400 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-15T18:56:04.400 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-01-15T18:56:04.400 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:56:04.401 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-01-15T18:56:04.413 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-01-15T18:56:04.457 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-01-15T18:56:04.479 INFO:teuthology.task.internal:Checking for old test directory... 2021-01-15T18:56:04.480 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2021-01-15T18:56:04.483 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2021-01-15T18:56:04.496 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-01-15T18:56:04.536 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-01-15T18:56:04.537 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2021-01-15T18:56:04.540 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2021-01-15T18:56:04.564 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-01-15T18:56:04.643 INFO:teuthology.run_tasks:Running task kernel... 2021-01-15T18:56:04.668 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'prometheus.a': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.b': {'kdb': True, 'sha1': 'distro'}} 2021-01-15T18:56:04.669 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-01-15T18:56:04.670 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2021-01-15T18:56:04.685 INFO:teuthology.orchestra.run.smithi001.stdout:4.18.0-193.28.1.el8_2.x86_64 2021-01-15T18:56:04.685 INFO:teuthology.task.kernel:Running kernel on smithi001: 4.18.0-193.28.1.el8_2.x86_64 2021-01-15T18:56:04.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum install -y kernel 2021-01-15T18:56:07.822 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS-8 - AppStream 28 MB/s | 6.3 MB 00:00 2021-01-15T18:56:08.982 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS-8 - Base 25 MB/s | 2.3 MB 00:00 2021-01-15T18:56:09.459 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS-8 - Extras 112 kB/s | 8.6 kB 00:00 2021-01-15T18:56:11.383 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 8.8 MB 00:01 2021-01-15T18:56:13.291 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 4.5 MB/s | 23 kB 00:00 2021-01-15T18:56:14.406 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-4.18.0-193.el8.x86_64 is already installed. 2021-01-15T18:56:14.406 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2021-01-15T18:56:14.589 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T18:56:14.595 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T18:56:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T18:56:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T18:56:14.597 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T18:56:14.598 INFO:teuthology.orchestra.run.smithi001.stdout: kernel x86_64 4.18.0-240.1.1.el8_3 CentOS-Base 4.3 M 2021-01-15T18:56:14.598 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2021-01-15T18:56:14.599 INFO:teuthology.orchestra.run.smithi001.stdout: linux-firmware noarch 20200619-101.git3890db36.el8_3 CentOS-Base 101 M 2021-01-15T18:56:14.599 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T18:56:14.599 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core x86_64 4.18.0-240.1.1.el8_3 CentOS-Base 30 M 2021-01-15T18:56:14.600 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules x86_64 4.18.0-240.1.1.el8_3 CentOS-Base 26 M 2021-01-15T18:56:14.600 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:56:14.600 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T18:56:14.601 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T18:56:14.601 INFO:teuthology.orchestra.run.smithi001.stdout:Install 3 Packages 2021-01-15T18:56:14.602 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 1 Package 2021-01-15T18:56:14.602 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:56:14.603 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 162 M 2021-01-15T18:56:14.603 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T18:56:14.723 INFO:teuthology.orchestra.run.smithi001.stdout:(1/4): kernel-4.18.0-240.1.1.el8_3.x86_64.rpm 34 MB/s | 4.3 MB 00:00 2021-01-15T18:56:15.842 INFO:teuthology.orchestra.run.smithi001.stdout:(2/4): kernel-core-4.18.0-240.1.1.el8_3.x86_64. 24 MB/s | 30 MB 00:01 2021-01-15T18:56:16.011 INFO:teuthology.orchestra.run.smithi001.stdout:(3/4): kernel-modules-4.18.0-240.1.1.el8_3.x86_ 18 MB/s | 26 MB 00:01 2021-01-15T18:56:17.259 INFO:teuthology.orchestra.run.smithi001.stdout:(4/4): linux-firmware-20200619-101.git3890db36. 40 MB/s | 101 MB 00:02 2021-01-15T18:56:17.262 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T18:56:17.262 INFO:teuthology.orchestra.run.smithi001.stdout:Total 61 MB/s | 162 MB 00:02 2021-01-15T18:56:17.359 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T18:56:17.673 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T18:56:17.673 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T18:56:19.379 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T18:56:19.380 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T18:56:30.049 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T18:56:31.917 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : linux-firmware-20200619-101.git3890db36.el8_3.noarch 1/5 2021-01-15T18:56:32.472 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-core-4.18.0-240.1.1.el8_3.x86_64 2/5 2021-01-15T18:56:35.309 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-4.18.0-240.1.1.el8_3.x86_64 2/5 2021-01-15T18:56:35.663 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-4.18.0-240.1.1.el8_3.x86_64 3/5 2021-01-15T18:56:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-4.18.0-240.1.1.el8_3.x86_64 3/5 2021-01-15T18:56:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-4.18.0-240.1.1.el8_3.x86_64 4/5 2021-01-15T18:56:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-01-15T18:57:00.236 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-4.18.0-240.1.1.el8_3.x86_64 5/5 2021-01-15T18:57:00.630 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-01-15T18:57:00.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-4.18.0-240.1.1.el8_3.x86_64 1/5 2021-01-15T18:57:00.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-core-4.18.0-240.1.1.el8_3.x86_64 2/5 2021-01-15T18:57:00.631 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-4.18.0-240.1.1.el8_3.x86_64 3/5 2021-01-15T18:57:00.632 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : linux-firmware-20200619-101.git3890db36.el8_3.noarch 4/5 2021-01-15T18:57:01.026 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-01-15T18:57:01.027 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:57:01.027 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2021-01-15T18:57:01.027 INFO:teuthology.orchestra.run.smithi001.stdout: linux-firmware-20200619-101.git3890db36.el8_3.noarch 2021-01-15T18:57:01.028 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:57:01.028 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T18:57:01.028 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:01.028 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:01.029 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:01.029 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:57:01.029 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T18:57:01.176 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel --last | head -n 1 2021-01-15T18:57:01.294 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-4.18.0-240.1.1.el8_3.x86_64 Fri 15 Jan 2021 06:56:38 PM UTC 2021-01-15T18:57:01.335 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:01.335 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-193.28.1.el8_2.x86_64 Expected: 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:01.336 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2021-01-15T18:57:01.354 INFO:teuthology.orchestra.run.smithi136.stdout:4.18.0-193.28.1.el8_2.x86_64 2021-01-15T18:57:01.354 INFO:teuthology.task.kernel:Running kernel on smithi136: 4.18.0-193.28.1.el8_2.x86_64 2021-01-15T18:57:01.354 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2021-01-15T18:57:04.142 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-8 - AppStream 23 MB/s | 6.3 MB 00:00 2021-01-15T18:57:05.312 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-8 - Base 15 MB/s | 2.3 MB 00:00 2021-01-15T18:57:05.768 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-8 - Extras 124 kB/s | 8.6 kB 00:00 2021-01-15T18:57:06.225 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.8 MB 00:00 2021-01-15T18:57:08.080 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 4.2 MB/s | 23 kB 00:00 2021-01-15T18:57:09.175 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-4.18.0-193.el8.x86_64 is already installed. 2021-01-15T18:57:09.176 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2021-01-15T18:57:09.361 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T18:57:09.373 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T18:57:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T18:57:09.374 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T18:57:09.374 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T18:57:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 4.18.0-240.1.1.el8_3 CentOS-Base 4.3 M 2021-01-15T18:57:09.375 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2021-01-15T18:57:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: linux-firmware noarch 20200619-101.git3890db36.el8_3 CentOS-Base 101 M 2021-01-15T18:57:09.375 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T18:57:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core x86_64 4.18.0-240.1.1.el8_3 CentOS-Base 30 M 2021-01-15T18:57:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules x86_64 4.18.0-240.1.1.el8_3 CentOS-Base 26 M 2021-01-15T18:57:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:09.376 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T18:57:09.377 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T18:57:09.377 INFO:teuthology.orchestra.run.smithi136.stdout:Install 3 Packages 2021-01-15T18:57:09.377 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 1 Package 2021-01-15T18:57:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:09.378 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 162 M 2021-01-15T18:57:09.378 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T18:57:09.507 INFO:teuthology.orchestra.run.smithi136.stdout:(1/4): kernel-4.18.0-240.1.1.el8_3.x86_64.rpm 33 MB/s | 4.3 MB 00:00 2021-01-15T18:57:10.548 INFO:teuthology.orchestra.run.smithi136.stdout:(2/4): kernel-core-4.18.0-240.1.1.el8_3.x86_64. 26 MB/s | 30 MB 00:01 2021-01-15T18:57:10.758 INFO:teuthology.orchestra.run.smithi136.stdout:(3/4): kernel-modules-4.18.0-240.1.1.el8_3.x86_ 18 MB/s | 26 MB 00:01 2021-01-15T18:57:12.106 INFO:teuthology.orchestra.run.smithi136.stdout:(4/4): linux-firmware-20200619-101.git3890db36. 39 MB/s | 101 MB 00:02 2021-01-15T18:57:12.109 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T18:57:12.110 INFO:teuthology.orchestra.run.smithi136.stdout:Total 59 MB/s | 162 MB 00:02 2021-01-15T18:57:12.201 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T18:57:12.501 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T18:57:12.501 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T18:57:14.443 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T18:57:14.445 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T18:57:24.338 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T18:57:26.064 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : linux-firmware-20200619-101.git3890db36.el8_3.noarch 1/5 2021-01-15T18:57:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-core-4.18.0-240.1.1.el8_3.x86_64 2/5 2021-01-15T18:57:29.247 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-4.18.0-240.1.1.el8_3.x86_64 2/5 2021-01-15T18:57:29.548 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-4.18.0-240.1.1.el8_3.x86_64 3/5 2021-01-15T18:57:32.421 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-4.18.0-240.1.1.el8_3.x86_64 3/5 2021-01-15T18:57:32.614 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-4.18.0-240.1.1.el8_3.x86_64 4/5 2021-01-15T18:57:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-01-15T18:57:52.713 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-4.18.0-240.1.1.el8_3.x86_64 5/5 2021-01-15T18:57:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-01-15T18:57:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-4.18.0-240.1.1.el8_3.x86_64 1/5 2021-01-15T18:57:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-core-4.18.0-240.1.1.el8_3.x86_64 2/5 2021-01-15T18:57:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-4.18.0-240.1.1.el8_3.x86_64 3/5 2021-01-15T18:57:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : linux-firmware-20200619-101.git3890db36.el8_3.noarch 4/5 2021-01-15T18:57:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-01-15T18:57:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:53.484 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2021-01-15T18:57:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: linux-firmware-20200619-101.git3890db36.el8_3.noarch 2021-01-15T18:57:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:53.484 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T18:57:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:53.485 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T18:57:53.637 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel --last | head -n 1 2021-01-15T18:57:53.744 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-4.18.0-240.1.1.el8_3.x86_64 Fri 15 Jan 2021 06:57:32 PM UTC 2021-01-15T18:57:53.746 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.746 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-193.28.1.el8_2.x86_64 Expected: 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.746 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-01-15T18:57:53.747 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-01-15T18:57:53.747 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-01-15T18:57:53.747 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2021-01-15T18:57:53.748 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=None, version=distro) 2021-01-15T18:57:53.748 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel --last | head -n 1 2021-01-15T18:57:53.850 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-4.18.0-240.1.1.el8_3.x86_64 Fri 15 Jan 2021 06:56:38 PM UTC 2021-01-15T18:57:53.852 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.852 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2021-01-15T18:57:53.885 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2021-01-15T18:57:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2021-01-15T18:57:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.02 2021-01-15T18:57:53.886 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 81.el8 2021-01-15T18:57:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2021-01-15T18:57:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Fri 13 Nov 2020 03:02:55 PM UTC 2021-01-15T18:57:53.887 INFO:teuthology.orchestra.run.smithi001.stdout:Group : System Environment/Base 2021-01-15T18:57:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 9192705 2021-01-15T18:57:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2021-01-15T18:57:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Tue 14 Apr 2020 03:18:37 PM UTC, Key ID 05b555b38483c65d 2021-01-15T18:57:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.02-81.el8.src.rpm 2021-01-15T18:57:53.888 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Tue 14 Apr 2020 02:54:04 PM UTC 2021-01-15T18:57:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : kbuilder.bsys.centos.org 2021-01-15T18:57:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:Relocations : (not relocatable) 2021-01-15T18:57:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : CentOS BuildSystem 2021-01-15T18:57:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2021-01-15T18:57:53.889 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2021-01-15T18:57:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2021-01-15T18:57:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2021-01-15T18:57:53.890 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:57:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-01-15T18:57:53.890 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2021-01-15T18:57:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2021-01-15T18:57:53.891 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2021-01-15T18:57:53.891 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T18:57:53.892 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2021-01-15T18:57:53.892 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-01-15T18:57:53.893 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:53.893 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-01-15T18:57:54.676 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2021-01-15T18:57:56.759 INFO:teuthology.orchestra.run.smithi001.stderr:done 2021-01-15T18:57:56.761 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2021-01-15T18:57:56.776 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.ARIGcQfr95 2021-01-15T18:57:56.776 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ARIGcQfr95 2021-01-15T18:57:56.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.ARIGcQfr95 2021-01-15T18:57:56.997 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.ARIGcQfr95 is 4KB 2021-01-15T18:57:57.046 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.ARIGcQfr95 2021-01-15T18:57:57.061 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2021-01-15T18:57:57.124 INFO:teuthology.orchestra.run.smithi001.stdout:2951a06464bf41f6a3bf91594147ecff-0-rescue.conf 2021-01-15T18:57:57.124 INFO:teuthology.orchestra.run.smithi001.stdout:2951a06464bf41f6a3bf91594147ecff-4.18.0-193.28.1.el8_2.x86_64.conf 2021-01-15T18:57:57.125 INFO:teuthology.orchestra.run.smithi001.stdout:2951a06464bf41f6a3bf91594147ecff-4.18.0-193.el8.x86_64.conf 2021-01-15T18:57:57.125 INFO:teuthology.orchestra.run.smithi001.stdout:2951a06464bf41f6a3bf91594147ecff-4.18.0-240.1.1.el8_3.x86_64.conf 2021-01-15T18:57:57.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default 2951a06464bf41f6a3bf91594147ecff-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:57.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2021-01-15T18:57:57.299 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2021-01-15T18:57:57.299 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2021-01-15T18:57:57.300 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel --last | head -n 1 2021-01-15T18:57:57.395 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-4.18.0-240.1.1.el8_3.x86_64 Fri 15 Jan 2021 06:57:32 PM UTC 2021-01-15T18:57:57.396 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:57.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2021-01-15T18:57:57.427 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2021-01-15T18:57:57.428 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2021-01-15T18:57:57.428 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.02 2021-01-15T18:57:57.428 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 81.el8 2021-01-15T18:57:57.428 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2021-01-15T18:57:57.429 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Fri 13 Nov 2020 03:02:55 PM UTC 2021-01-15T18:57:57.429 INFO:teuthology.orchestra.run.smithi136.stdout:Group : System Environment/Base 2021-01-15T18:57:57.429 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 9192705 2021-01-15T18:57:57.429 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2021-01-15T18:57:57.429 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Tue 14 Apr 2020 03:18:37 PM UTC, Key ID 05b555b38483c65d 2021-01-15T18:57:57.430 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.02-81.el8.src.rpm 2021-01-15T18:57:57.430 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Tue 14 Apr 2020 02:54:04 PM UTC 2021-01-15T18:57:57.430 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : kbuilder.bsys.centos.org 2021-01-15T18:57:57.430 INFO:teuthology.orchestra.run.smithi136.stdout:Relocations : (not relocatable) 2021-01-15T18:57:57.430 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : CentOS BuildSystem 2021-01-15T18:57:57.431 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : CentOS 2021-01-15T18:57:57.431 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2021-01-15T18:57:57.431 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2021-01-15T18:57:57.431 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2021-01-15T18:57:57.431 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:57.432 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-01-15T18:57:57.432 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2021-01-15T18:57:57.432 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2021-01-15T18:57:57.432 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2021-01-15T18:57:57.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T18:57:57.433 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2021-01-15T18:57:57.433 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-01-15T18:57:57.434 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:57:57.434 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-01-15T18:57:59.098 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2021-01-15T18:58:01.464 INFO:teuthology.orchestra.run.smithi136.stderr:done 2021-01-15T18:58:01.466 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2021-01-15T18:58:01.479 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.wXH422gZ23 2021-01-15T18:58:01.480 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wXH422gZ23 2021-01-15T18:58:01.549 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.wXH422gZ23 2021-01-15T18:58:01.704 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.wXH422gZ23 is 4KB 2021-01-15T18:58:01.752 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.wXH422gZ23 2021-01-15T18:58:01.766 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2021-01-15T18:58:01.830 INFO:teuthology.orchestra.run.smithi136.stdout:2951a06464bf41f6a3bf91594147ecff-0-rescue.conf 2021-01-15T18:58:01.830 INFO:teuthology.orchestra.run.smithi136.stdout:2951a06464bf41f6a3bf91594147ecff-4.18.0-193.28.1.el8_2.x86_64.conf 2021-01-15T18:58:01.830 INFO:teuthology.orchestra.run.smithi136.stdout:2951a06464bf41f6a3bf91594147ecff-4.18.0-193.el8.x86_64.conf 2021-01-15T18:58:01.831 INFO:teuthology.orchestra.run.smithi136.stdout:2951a06464bf41f6a3bf91594147ecff-4.18.0-240.1.1.el8_3.x86_64.conf 2021-01-15T18:58:01.832 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 2951a06464bf41f6a3bf91594147ecff-4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:58:01.994 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2021-01-15T18:58:32.024 INFO:teuthology.misc:Re-opening connections... 2021-01-15T18:58:32.025 INFO:teuthology.misc:trying to connect to ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T18:58:32.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T18:58:32.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:59:32.030 DEBUG:teuthology.orchestra.remote:timed out 2021-01-15T18:59:32.032 INFO:teuthology.misc:trying to connect to ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T18:59:32.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T18:59:32.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:59:39.614 DEBUG:teuthology.orchestra.run.smithi136:> true 2021-01-15T18:59:39.944 DEBUG:teuthology.misc:waited 67.91849374771118 2021-01-15T18:59:40.945 INFO:teuthology.misc:trying to connect to ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T18:59:40.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T18:59:40.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T18:59:41.208 DEBUG:teuthology.orchestra.run.smithi001:> true 2021-01-15T18:59:41.530 DEBUG:teuthology.misc:waited 69.5049614906311 2021-01-15T18:59:42.532 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2021-01-15T18:59:42.532 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-240.1.1.el8_3.x86_64"... 2021-01-15T18:59:42.533 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2021-01-15T18:59:42.550 INFO:teuthology.orchestra.run.smithi001.stdout:4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:59:42.550 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-240.1.1.el8_3.x86_64 vs 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:59:42.551 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-01-15T18:59:42.551 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2021-01-15T18:59:42.551 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2021-01-15T18:59:42.552 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-240.1.1.el8_3.x86_64"... 2021-01-15T18:59:42.552 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2021-01-15T18:59:42.569 INFO:teuthology.orchestra.run.smithi136.stdout:4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:59:42.569 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-240.1.1.el8_3.x86_64 vs 4.18.0-240.1.1.el8_3.x86_64 2021-01-15T18:59:42.570 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-01-15T18:59:42.570 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2021-01-15T18:59:43.572 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-01-15T18:59:43.572 DEBUG:teuthology.orchestra.run.smithi001:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-01-15T18:59:43.649 INFO:teuthology.orchestra.run.smithi001.stdout:ttyS1 2021-01-15T18:59:43.669 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-01-15T18:59:43.670 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-01-15T18:59:43.763 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2021-01-15T18:59:43.795 INFO:teuthology.run_tasks:Running task internal.base... 2021-01-15T18:59:43.816 INFO:teuthology.task.internal:Creating test directory... 2021-01-15T18:59:43.816 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-01-15T18:59:43.833 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-01-15T18:59:43.866 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-01-15T18:59:43.893 INFO:teuthology.run_tasks:Running task internal.archive... 2021-01-15T18:59:43.923 INFO:teuthology.task.internal:Creating archive directory... 2021-01-15T18:59:43.923 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-01-15T18:59:43.933 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-01-15T18:59:43.969 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-01-15T18:59:44.006 INFO:teuthology.task.internal:Enabling coredump saving... 2021-01-15T18:59:44.007 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-01-15T18:59:44.011 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-01-15T18:59:44.041 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-15T18:59:44.046 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-15T18:59:44.054 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-15T18:59:44.058 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-01-15T18:59:44.059 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-01-15T18:59:44.070 INFO:teuthology.task.internal:Configuring sudo... 2021-01-15T18:59:44.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-01-15T18:59:44.109 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-01-15T18:59:44.141 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-01-15T18:59:44.173 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-01-15T18:59:44.174 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-01-15T18:59:44.185 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-01-15T18:59:44.202 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-01-15T18:59:44.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2021-01-15T18:59:44.355 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-01-15T18:59:44.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2021-01-15T18:59:44.496 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T18:59:44.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-01-15T18:59:44.573 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-01-15T18:59:44.625 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2021-01-15T18:59:44.673 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-01-15T18:59:44.774 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2021-01-15T18:59:44.811 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T18:59:44.811 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-01-15T18:59:44.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2021-01-15T18:59:44.891 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2021-01-15T18:59:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-01-15T18:59:44.997 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-01-15T18:59:45.270 INFO:teuthology.run_tasks:Running task internal.timer... 2021-01-15T18:59:45.283 INFO:teuthology.task.internal:Starting timer... 2021-01-15T18:59:45.283 INFO:teuthology.run_tasks:Running task pcp... 2021-01-15T18:59:45.347 INFO:teuthology.run_tasks:Running task selinux... 2021-01-15T18:59:45.461 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2021-01-15T18:59:45.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2021-01-15T18:59:45.543 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: [ OK ] 2021-01-15T18:59:45.544 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2021-01-15T18:59:45.653 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: [ OK ] 2021-01-15T18:59:45.653 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-01-15T18:59:45.655 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2021-01-15T18:59:45.682 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2021-01-15T18:59:45.683 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2021-01-15T18:59:45.719 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2021-01-15T18:59:45.720 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2021-01-15T18:59:45.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2021-01-15T18:59:45.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-15T18:59:45.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2021-01-15T18:59:45.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-01-15T18:59:45.786 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-01-15T18:59:45.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2021-01-15T18:59:45.835 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2021-01-15T18:59:45.862 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-01-15T18:59:45.872 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2021-01-15T18:59:45.873 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2021-01-15T18:59:45.885 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-01-15T18:59:45.894 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-01-15T18:59:46.252 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-01-15T18:59:47.134 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-01-15T18:59:49.666 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-01-15T18:59:49.685 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:49 +0000 (0:00:00.070) 0:00:00.070 ******** 2021-01-15T18:59:49.699 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:49 +0000 (0:00:00.013) 0:00:00.083 ******** 2021-01-15T18:59:51.285 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi001.front.sepia.ceph.com] 2021-01-15T18:59:51.332 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] 2021-01-15T18:59:51.369 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:51 +0000 (0:00:01.670) 0:00:01.754 ******** 2021-01-15T18:59:51.435 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:51 +0000 (0:00:00.065) 0:00:01.820 ******** 2021-01-15T18:59:51.686 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-01-15T18:59:51.690 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-01-15T18:59:51.719 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:51 +0000 (0:00:00.283) 0:00:02.104 ******** 2021-01-15T18:59:51.729 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-01-15T18:59:51.764 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:51 +0000 (0:00:00.045) 0:00:02.149 ******** 2021-01-15T18:59:52.119 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-01-15T18:59:52.121 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-01-15T18:59:52.157 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:52 +0000 (0:00:00.393) 0:00:02.542 ******** 2021-01-15T18:59:52.165 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-01-15T18:59:52.204 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:52 +0000 (0:00:00.046) 0:00:02.588 ******** 2021-01-15T18:59:52.211 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-01-15T18:59:52.212 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:52.245 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:52 +0000 (0:00:00.041) 0:00:02.630 ******** 2021-01-15T18:59:52.257 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:52.298 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:52 +0000 (0:00:00.047) 0:00:02.677 ******** 2021-01-15T18:59:52.568 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T18:59:52.603 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:52 +0000 (0:00:00.310) 0:00:02.987 ******** 2021-01-15T18:59:52.610 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T18:59:52.653 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:52 +0000 (0:00:00.050) 0:00:03.038 ******** 2021-01-15T18:59:53.602 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737192.6173508-14028254549514/source", "state": "file", "uid": 0} 2021-01-15T18:59:53.636 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:53 +0000 (0:00:00.982) 0:00:04.021 ******** 2021-01-15T18:59:53.643 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737192.6673563-93678234578745/source", "state": "file", "uid": 0} 2021-01-15T18:59:53.682 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:53 +0000 (0:00:00.045) 0:00:04.066 ******** 2021-01-15T18:59:54.028 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-01-15T18:59:54.030 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-01-15T18:59:54.032 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-01-15T18:59:54.065 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-01-15T18:59:54.067 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-01-15T18:59:54.182 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-01-15T18:59:54.189 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-01-15T18:59:54.337 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:54 +0000 (0:00:00.655) 0:00:04.722 ******** 2021-01-15T18:59:54.368 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:54 +0000 (0:00:00.030) 0:00:04.753 ******** 2021-01-15T18:59:55.009 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-01-15T18:59:55.012 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] 2021-01-15T18:59:55.022 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] 2021-01-15T18:59:55.157 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-01-15T18:59:55.158 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:55 +0000 (0:00:00.789) 0:00:05.542 ******** 2021-01-15T18:59:55.214 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:55 +0000 (0:00:00.056) 0:00:05.599 ******** 2021-01-15T18:59:55.774 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-01-15T18:59:55.776 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.161954", "end": "2021-01-15 18:59:55.747480", "rc": 0, "start": "2021-01-15 18:59:55.585526", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-01-15T18:59:55.812 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:55 +0000 (0:00:00.597) 0:00:06.197 ******** 2021-01-15T18:59:55.819 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.187189", "end": "2021-01-15 18:59:55.758618", "rc": 0, "start": "2021-01-15 18:59:55.571429", "stderr": "", "stderr_lines": [], "stdout": "0caff10a478f2674721da1965013b225f3ea5800", "stdout_lines": ["0caff10a478f2674721da1965013b225f3ea5800"]} 2021-01-15T18:59:55.856 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:55 +0000 (0:00:00.044) 0:00:06.241 ******** 2021-01-15T18:59:56.042 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.014320", "end": "2021-01-15 18:59:56.016773", "failed_when_result": false, "rc": 0, "start": "2021-01-15 18:59:56.002453", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2021-01-15T18:59:56.099 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.242) 0:00:06.483 ******** 2021-01-15T18:59:56.109 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013611", "end": "2021-01-15 18:59:56.038129", "failed_when_result": false, "rc": 0, "start": "2021-01-15 18:59:56.024518", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2021-01-15T18:59:56.152 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.052) 0:00:06.536 ******** 2021-01-15T18:59:56.158 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2021-01-15T18:59:56.159 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.193 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.041) 0:00:06.578 ******** 2021-01-15T18:59:56.200 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.245 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.051) 0:00:06.629 ******** 2021-01-15T18:59:56.250 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2021-01-15T18:59:56.252 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.287 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.042) 0:00:06.672 ******** 2021-01-15T18:59:56.294 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.329 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.041) 0:00:06.714 ******** 2021-01-15T18:59:56.335 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2021-01-15T18:59:56.336 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.370 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.040) 0:00:06.754 ******** 2021-01-15T18:59:56.376 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.411 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.041) 0:00:06.796 ******** 2021-01-15T18:59:56.418 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.452 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.040) 0:00:06.837 ******** 2021-01-15T18:59:56.458 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.460 INFO:teuthology.task.ansible.out: 2021-01-15T18:59:56.494 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.041) 0:00:06.878 ******** 2021-01-15T18:59:56.500 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2021-01-15T18:59:56.502 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.538 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.044) 0:00:06.922 ******** 2021-01-15T18:59:56.546 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T18:59:56.582 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.041) 0:00:06.964 ******** 2021-01-15T18:59:56.745 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-01-15T18:59:56.747 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T18:59:56.779 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.200) 0:00:07.164 ******** 2021-01-15T18:59:56.787 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T18:59:56.823 INFO:teuthology.task.ansible.out:Friday 15 January 2021 18:59:56 +0000 (0:00:00.043) 0:00:07.208 ******** 2021-01-15T18:59:57.023 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-01-15T18:59:57.025 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-01-15T18:59:57.067 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-01-15T18:59:57.224 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-01-15T18:59:57.257 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-01-15T18:59:57.418 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-01-15T18:59:57.451 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-01-15T18:59:57.622 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-01-15T18:59:57.644 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-01-15T18:59:57.827 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-01-15T18:59:57.836 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-01-15T18:59:58.029 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-01-15T18:59:58.031 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-01-15T18:59:58.219 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-01-15T18:59:58.240 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-01-15T18:59:58.409 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-01-15T18:59:58.431 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-01-15T18:59:58.615 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-01-15T18:59:58.637 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-01-15T18:59:58.810 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-01-15T18:59:58.849 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-01-15T18:59:58.998 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-01-15T18:59:59.062 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-01-15T18:59:59.190 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-01-15T18:59:59.264 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-01-15T18:59:59.380 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-01-15T18:59:59.466 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-01-15T18:59:59.572 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-01-15T18:59:59.657 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-01-15T18:59:59.763 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-01-15T18:59:59.858 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-01-15T18:59:59.954 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-01-15T19:00:00.055 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-01-15T19:00:00.139 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-01-15T19:00:00.257 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-01-15T19:00:00.331 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-01-15T19:00:00.457 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-01-15T19:00:00.521 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-01-15T19:00:00.658 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-01-15T19:00:00.721 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-01-15T19:00:00.858 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-01-15T19:00:00.912 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-01-15T19:00:01.060 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-01-15T19:00:01.103 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-01-15T19:00:01.262 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-01-15T19:00:01.296 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-01-15T19:00:01.459 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-01-15T19:00:01.504 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-01-15T19:00:01.664 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-01-15T19:00:01.696 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-01-15T19:00:01.867 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-01-15T19:00:01.900 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-01-15T19:00:02.069 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-01-15T19:00:02.115 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-01-15T19:00:02.277 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-01-15T19:00:02.300 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-01-15T19:00:02.474 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-01-15T19:00:02.501 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-01-15T19:00:02.681 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-01-15T19:00:02.713 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-01-15T19:00:02.887 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-01-15T19:00:02.899 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-01-15T19:00:03.095 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-01-15T19:00:03.097 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-01-15T19:00:03.306 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-01-15T19:00:03.307 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-01-15T19:00:03.510 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-01-15T19:00:03.699 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-01-15T19:00:03.711 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-01-15T19:00:03.902 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-01-15T19:00:03.950 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-01-15T19:00:04.100 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-01-15T19:00:04.132 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-01-15T19:00:04.288 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-01-15T19:00:04.355 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-01-15T19:00:04.493 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-01-15T19:00:04.548 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-01-15T19:00:04.684 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-15T19:00:04.751 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-01-15T19:00:04.878 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-01-15T19:00:04.946 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-15T19:00:05.062 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-01-15T19:00:05.174 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-01-15T19:00:05.310 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-01-15T19:00:05.378 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-01-15T19:00:05.533 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-01-15T19:00:05.612 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-01-15T19:00:05.739 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-01-15T19:00:05.837 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-01-15T19:00:05.960 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-01-15T19:00:06.088 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-01-15T19:00:06.184 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-01-15T19:00:06.311 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-01-15T19:00:06.391 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-15T19:00:06.530 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-01-15T19:00:06.585 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-01-15T19:00:06.751 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-15T19:00:06.784 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-01-15T19:00:06.971 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-01-15T19:00:06.997 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-01-15T19:00:07.182 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-01-15T19:00:07.213 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-01-15T19:00:07.422 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-01-15T19:00:07.627 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-01-15T19:00:07.628 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-01-15T19:00:07.844 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-01-15T19:00:07.846 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-01-15T19:00:08.049 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-01-15T19:00:08.075 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-01-15T19:00:08.240 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-01-15T19:00:08.273 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-01-15T19:00:08.443 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-01-15T19:00:08.476 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-01-15T19:00:08.638 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-01-15T19:00:08.716 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-01-15T19:00:08.848 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-01-15T19:00:08.928 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-01-15T19:00:09.036 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-01-15T19:00:09.121 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-01-15T19:00:09.228 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-01-15T19:00:09.330 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-01-15T19:00:09.443 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-01-15T19:00:09.529 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-01-15T19:00:09.637 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-01-15T19:00:09.746 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-01-15T19:00:09.842 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-15T19:00:09.961 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-01-15T19:00:10.047 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-01-15T19:00:10.164 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-01-15T19:00:10.240 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-01-15T19:00:10.368 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-01-15T19:00:10.433 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-01-15T19:00:10.562 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-01-15T19:00:10.626 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-01-15T19:00:10.777 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-01-15T19:00:10.821 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-01-15T19:00:10.968 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-01-15T19:00:11.012 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-01-15T19:00:11.179 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-01-15T19:00:11.214 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-01-15T19:00:11.386 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-01-15T19:00:11.408 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-01-15T19:00:11.590 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-01-15T19:00:11.602 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-01-15T19:00:11.793 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-01-15T19:00:11.794 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-01-15T19:00:11.986 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-01-15T19:00:11.989 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-01-15T19:00:12.172 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-01-15T19:00:12.194 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-01-15T19:00:12.364 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-01-15T19:00:12.397 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-01-15T19:00:12.559 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-01-15T19:00:12.620 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-01-15T19:00:12.753 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-01-15T19:00:12.808 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-01-15T19:00:12.947 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-01-15T19:00:13.022 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-01-15T19:00:13.138 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-01-15T19:00:13.216 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-01-15T19:00:13.331 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-01-15T19:00:13.417 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-01-15T19:00:13.536 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-01-15T19:00:13.624 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-01-15T19:00:13.734 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-01-15T19:00:13.833 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-01-15T19:00:13.932 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-01-15T19:00:14.040 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-01-15T19:00:14.127 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-01-15T19:00:14.248 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-01-15T19:00:14.325 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-01-15T19:00:14.468 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-01-15T19:00:14.532 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-01-15T19:00:14.671 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-01-15T19:00:14.731 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-01-15T19:00:14.878 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-01-15T19:00:14.932 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-01-15T19:00:15.082 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-01-15T19:00:15.139 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-01-15T19:00:15.290 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-01-15T19:00:15.335 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-01-15T19:00:15.500 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-01-15T19:00:15.545 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-01-15T19:00:15.710 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-01-15T19:00:15.757 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-01-15T19:00:15.926 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-01-15T19:00:15.960 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-01-15T19:00:16.132 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-01-15T19:00:16.162 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-01-15T19:00:16.366 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-01-15T19:00:16.378 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-01-15T19:00:16.571 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-01-15T19:00:16.574 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-01-15T19:00:16.793 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-01-15T19:00:16.998 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-01-15T19:00:17.010 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-01-15T19:00:17.202 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-01-15T19:00:17.213 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-01-15T19:00:17.394 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-15T19:00:17.416 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-01-15T19:00:17.588 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-01-15T19:00:17.620 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-01-15T19:00:17.781 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-01-15T19:00:17.827 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-01-15T19:00:17.978 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-01-15T19:00:18.050 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-01-15T19:00:18.188 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-01-15T19:00:18.251 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-01-15T19:00:18.379 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-01-15T19:00:18.455 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-01-15T19:00:18.583 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-01-15T19:00:18.656 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-01-15T19:00:18.782 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-01-15T19:00:18.859 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-01-15T19:00:18.976 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-01-15T19:00:19.083 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-01-15T19:00:19.180 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-01-15T19:00:19.287 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-01-15T19:00:19.381 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-01-15T19:00:19.486 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-01-15T19:00:19.570 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-01-15T19:00:19.685 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-01-15T19:00:19.780 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-01-15T19:00:19.898 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-01-15T19:00:19.965 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-01-15T19:00:20.107 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-01-15T19:00:20.176 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-01-15T19:00:20.313 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-01-15T19:00:20.366 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-01-15T19:00:20.513 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-01-15T19:00:20.571 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-01-15T19:00:20.730 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-01-15T19:00:20.762 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-01-15T19:00:20.932 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-01-15T19:00:20.954 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-01-15T19:00:21.134 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-01-15T19:00:21.168 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-01-15T19:00:21.335 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-01-15T19:00:21.346 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-01-15T19:00:21.541 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-01-15T19:00:21.543 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-01-15T19:00:21.733 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-01-15T19:00:21.745 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-01-15T19:00:21.927 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-01-15T19:00:21.950 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-01-15T19:00:22.113 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-01-15T19:00:22.157 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-01-15T19:00:22.311 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-01-15T19:00:22.362 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-01-15T19:00:22.500 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-01-15T19:00:22.575 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-01-15T19:00:22.691 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-01-15T19:00:22.776 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-01-15T19:00:22.882 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-01-15T19:00:22.991 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-01-15T19:00:23.088 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-01-15T19:00:23.184 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-01-15T19:00:23.280 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-01-15T19:00:23.391 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-01-15T19:00:23.478 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-01-15T19:00:23.587 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-01-15T19:00:23.683 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-01-15T19:00:23.800 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-01-15T19:00:23.861 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-01-15T19:00:24.000 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-01-15T19:00:24.204 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-01-15T19:00:24.405 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-01-15T19:00:24.608 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-01-15T19:00:24.809 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-01-15T19:00:25.009 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-01-15T19:00:25.267 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10102, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-01-15T19:00:26.596 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10102, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-01-15T19:00:26.618 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-01-15T19:00:27.907 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-01-15T19:00:28.110 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:28 +0000 (0:00:31.286) 0:00:38.494 ******** 2021-01-15T19:00:28.116 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "name": "adking", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-01-15T19:00:28.153 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-01-15T19:00:28.187 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:28 +0000 (0:00:00.077) 0:00:38.572 ******** 2021-01-15T19:00:28.269 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-01-15T19:00:28.270 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-01-15T19:00:28.272 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-01-15T19:00:28.274 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-01-15T19:00:28.276 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkK 2021-01-15T19:00:28.277 INFO:teuthology.task.ansible.out:ztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desk 2021-01-15T19:00:28.278 INFO:teuthology.task.ansible.out:top XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 2021-01-15T19:00:28.280 INFO:teuthology.task.ansible.out:HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg 2021-01-15T19:00:28.282 INFO:teuthology.task.ansible.out:e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df 2021-01-15T19:00:28.286 INFO:teuthology.task.ansible.out:79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hm 2021-01-15T19:00:28.287 INFO:teuthology.task.ansible.out:p+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b 2021-01-15T19:00:28.289 INFO:teuthology.task.ansible.out:0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAM 2021-01-15T19:00:28.291 INFO:teuthology.task.ansible.out:Xq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jel 2021-01-15T19:00:28.293 INFO:teuthology.task.ansible.out:opez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf 2021-01-15T19:00:28.296 INFO:teuthology.task.ansible.out:0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}]}, "changed": false} 2021-01-15T19:00:28.332 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:28 +0000 (0:00:00.144) 0:00:38.716 ******** 2021-01-15T19:00:29.503 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi136.front.sepia.ceph.com -> localhost] => {"after": "0caff10a478f2674721da1965013b225f3ea5800", "attempts": 1, "before": "0caff10a478f2674721da1965013b225f3ea5800", "changed": false, "remote_url_changed": false} 2021-01-15T19:00:29.538 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:29 +0000 (0:00:01.206) 0:00:39.922 ******** 2021-01-15T19:00:29.664 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2021-01-15T19:00:29.666 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "name": "amathuri", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-01-15T19:00:29.796 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:29 +0000 (0:00:00.257) 0:00:40.180 ******** 2021-01-15T19:00:29.823 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2021-01-15T19:00:29.858 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:29 +0000 (0:00:00.062) 0:00:40.242 ******** 2021-01-15T19:00:29.865 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:00:29.867 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-01-15T19:00:29.920 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2021-01-15T19:00:29.921 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-01-15T19:00:29.923 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-01-15T19:00:29.925 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancic 2021-01-15T19:00:29.926 INFO:teuthology.task.ansible.out:h", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkK 2021-01-15T19:00:29.928 INFO:teuthology.task.ansible.out:ztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 2021-01-15T19:00:29.929 INFO:teuthology.task.ansible.out:HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg 2021-01-15T19:00:29.931 INFO:teuthology.task.ansible.out:e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6 2021-01-15T19:00:29.931 INFO:teuthology.task.ansible.out:d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df 2021-01-15T19:00:29.933 INFO:teuthology.task.ansible.out:79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hm 2021-01-15T19:00:29.934 INFO:teuthology.task.ansible.out:p+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b 2021-01-15T19:00:29.936 INFO:teuthology.task.ansible.out:0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAM 2021-01-15T19:00:29.938 INFO:teuthology.task.ansible.out:Xq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jel 2021-01-15T19:00:29.939 INFO:teuthology.task.ansible.out:opez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf 2021-01-15T19:00:29.941 INFO:teuthology.task.ansible.out:0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4b 2021-01-15T19:00:29.942 INFO:teuthology.task.ansible.out:ea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}]}, "changed": false} 2021-01-15T19:00:29.982 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:29 +0000 (0:00:00.124) 0:00:40.367 ******** 2021-01-15T19:00:30.051 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:00:30.053 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-01-15T19:00:30.273 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-01-15T19:00:30.483 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-01-15T19:00:30.485 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-01-15T19:00:30.579 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* ok: [smithi001.front.sepia.ceph.com -> localhost] => {"after": "0caff10a478f2674721da1965013b225f3ea5800", "attempts": 1, "before": "0caff10a478f2674721da1965013b225f3ea5800", "changed": false, "remote_url_changed": false} 2021-01-15T19:00:30.612 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:30 +0000 (0:00:00.629) 0:00:40.996 ******** 2021-01-15T19:00:30.701 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path 2021-01-15T19:00:30.703 INFO:teuthology.task.ansible.out:": null, "state": "present", "user": "sage", "validate_certs": true} 2021-01-15T19:00:30.918 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-01-15T19:00:30.921 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-01-15T19:00:30.923 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-01-15T19:00:31.134 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-01-15T19:00:31.136 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-01-15T19:00:31.348 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-01-15T19:00:31.370 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-01-15T19:00:31.558 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-01-15T19:00:31.569 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:31.601 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-01-15T19:00:31.603 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-01-15T19:00:31.782 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-01-15T19:00:31.783 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-01-15T19:00:31.825 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-01-15T19:00:31.986 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-01-15T19:00:31.988 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-01-15T19:00:31.997 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:32.009 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:32.052 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-01-15T19:00:32.053 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-01-15T19:00:32.221 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-01-15T19:00:32.286 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-01-15T19:00:32.447 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-01-15T19:00:32.450 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-01-15T19:00:32.522 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-01-15T19:00:32.672 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-01-15T19:00:32.747 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-01-15T19:00:32.759 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:32.897 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-01-15T19:00:32.994 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-01-15T19:00:32.996 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-01-15T19:00:33.112 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-01-15T19:00:33.218 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-01-15T19:00:33.220 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-01-15T19:00:33.229 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:33.250 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:33.323 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-01-15T19:00:33.470 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-01-15T19:00:33.543 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-01-15T19:00:33.713 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-01-15T19:00:33.715 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-01-15T19:00:33.757 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2021-01-15T19:00:33.948 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-01-15T19:00:33.993 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-01-15T19:00:33.995 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-01-15T19:00:34.174 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-01-15T19:00:34.217 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-01-15T19:00:34.219 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-01-15T19:00:34.401 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-01-15T19:00:34.434 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-01-15T19:00:34.635 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-01-15T19:00:34.659 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-01-15T19:00:34.861 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-01-15T19:00:34.883 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-01-15T19:00:34.887 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-01-15T19:00:35.079 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2021-01-15T19:00:35.113 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-01-15T19:00:35.317 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-01-15T19:00:35.319 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-01-15T19:00:35.328 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-01-15T19:00:35.541 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-01-15T19:00:35.543 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-01-15T19:00:35.553 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-01-15T19:00:35.759 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-01-15T19:00:35.770 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-01-15T19:00:35.982 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-01-15T19:00:36.005 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-01-15T19:00:36.208 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-01-15T19:00:36.210 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-01-15T19:00:36.231 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-01-15T19:00:36.233 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-01-15T19:00:36.432 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-01-15T19:00:36.455 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-01-15T19:00:36.650 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-01-15T19:00:36.683 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-01-15T19:00:36.862 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-01-15T19:00:36.916 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-01-15T19:00:37.074 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-01-15T19:00:37.140 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-01-15T19:00:37.287 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-01-15T19:00:37.361 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-01-15T19:00:37.498 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-01-15T19:00:37.593 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-01-15T19:00:37.595 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-01-15T19:00:37.720 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-15T19:00:37.827 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-01-15T19:00:37.945 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-01-15T19:00:38.051 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-01-15T19:00:38.180 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-01-15T19:00:38.288 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-01-15T19:00:38.394 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-01-15T19:00:38.396 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-01-15T19:00:38.510 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-01-15T19:00:38.617 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-01-15T19:00:38.620 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-01-15T19:00:38.735 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-01-15T19:00:38.831 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-01-15T19:00:38.968 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-01-15T19:00:39.042 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-01-15T19:00:39.202 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-15T19:00:39.247 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-01-15T19:00:39.430 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-01-15T19:00:39.463 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-01-15T19:00:39.465 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-15T19:00:39.655 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-01-15T19:00:39.699 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-01-15T19:00:39.701 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-01-15T19:00:39.702 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-01-15T19:00:39.879 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-01-15T19:00:39.881 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-01-15T19:00:39.912 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-01-15T19:00:40.105 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-01-15T19:00:40.108 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-01-15T19:00:40.141 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-01-15T19:00:40.345 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-01-15T19:00:40.347 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-01-15T19:00:40.567 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-01-15T19:00:40.569 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-01-15T19:00:40.781 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-01-15T19:00:40.793 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-01-15T19:00:40.991 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-01-15T19:00:40.993 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-01-15T19:00:41.013 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-01-15T19:00:41.015 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-15T19:00:41.199 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-01-15T19:00:41.231 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-01-15T19:00:41.233 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-01-15T19:00:41.234 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-01-15T19:00:41.408 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-01-15T19:00:41.450 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-01-15T19:00:41.618 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-01-15T19:00:41.620 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-01-15T19:00:41.671 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-01-15T19:00:41.830 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-01-15T19:00:41.893 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-01-15T19:00:42.039 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-01-15T19:00:42.112 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-01-15T19:00:42.246 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-01-15T19:00:42.248 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-01-15T19:00:42.331 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-01-15T19:00:42.459 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-01-15T19:00:42.461 INFO:teuthology.task.ansible.out: 2021-01-15T19:00:42.554 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-01-15T19:00:42.556 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-01-15T19:00:42.681 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-01-15T19:00:42.777 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-01-15T19:00:42.881 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-01-15T19:00:42.988 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-01-15T19:00:43.092 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-15T19:00:43.209 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-01-15T19:00:43.211 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-01-15T19:00:43.314 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-01-15T19:00:43.440 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-01-15T19:00:43.524 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-01-15T19:00:43.661 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-01-15T19:00:43.736 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-01-15T19:00:43.885 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-01-15T19:00:43.887 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-01-15T19:00:43.970 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-01-15T19:00:44.118 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-01-15T19:00:44.173 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-01-15T19:00:44.331 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-01-15T19:00:44.384 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-01-15T19:00:44.554 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-01-15T19:00:44.586 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-01-15T19:00:44.769 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-01-15T19:00:44.802 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-01-15T19:00:44.992 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-01-15T19:00:45.014 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-01-15T19:00:45.216 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-01-15T19:00:45.226 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-01-15T19:00:45.425 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-01-15T19:00:45.447 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-01-15T19:00:45.635 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-01-15T19:00:45.667 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-01-15T19:00:45.847 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-01-15T19:00:45.890 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-01-15T19:00:46.090 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-01-15T19:00:46.111 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-01-15T19:00:46.301 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-01-15T19:00:46.366 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-01-15T19:00:46.524 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-01-15T19:00:46.577 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-01-15T19:00:46.736 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-01-15T19:00:46.738 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-01-15T19:00:46.810 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-01-15T19:00:46.945 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-01-15T19:00:47.029 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-01-15T19:00:47.165 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-01-15T19:00:47.260 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-01-15T19:00:47.378 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-01-15T19:00:47.493 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-01-15T19:00:47.597 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-01-15T19:00:47.712 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-01-15T19:00:47.806 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-01-15T19:00:47.808 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-01-15T19:00:47.931 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-01-15T19:00:48.015 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-01-15T19:00:48.017 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-01-15T19:00:48.162 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-01-15T19:00:48.225 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-01-15T19:00:48.385 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-01-15T19:00:48.438 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-01-15T19:00:48.605 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-01-15T19:00:48.607 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-01-15T19:00:48.647 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-01-15T19:00:48.824 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-01-15T19:00:48.856 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-01-15T19:00:49.047 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-01-15T19:00:49.070 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-01-15T19:00:49.263 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-01-15T19:00:49.296 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-01-15T19:00:49.494 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-01-15T19:00:49.515 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-01-15T19:00:49.715 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-01-15T19:00:49.716 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-01-15T19:00:49.726 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-01-15T19:00:49.728 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-01-15T19:00:49.936 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-01-15T19:00:49.938 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-01-15T19:00:50.146 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-01-15T19:00:50.158 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-01-15T19:00:50.370 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-01-15T19:00:50.382 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-01-15T19:00:50.572 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-01-15T19:00:50.604 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-01-15T19:00:50.793 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-01-15T19:00:50.830 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-01-15T19:00:51.002 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-01-15T19:00:51.064 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-01-15T19:00:51.224 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-01-15T19:00:51.226 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-15T19:00:51.300 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-01-15T19:00:51.439 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-01-15T19:00:51.524 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-01-15T19:00:51.664 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-01-15T19:00:51.740 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-01-15T19:00:51.744 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-01-15T19:00:51.877 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-01-15T19:00:51.966 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-01-15T19:00:52.080 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-01-15T19:00:52.186 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-01-15T19:00:52.301 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-01-15T19:00:52.416 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-01-15T19:00:52.521 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-01-15T19:00:52.638 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-01-15T19:00:52.741 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-01-15T19:00:52.857 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-01-15T19:00:52.954 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-01-15T19:00:53.081 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-01-15T19:00:53.165 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-01-15T19:00:53.302 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-01-15T19:00:53.303 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-01-15T19:00:53.375 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-01-15T19:00:53.512 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-01-15T19:00:53.574 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-01-15T19:00:53.741 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-01-15T19:00:53.784 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-01-15T19:00:53.961 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-01-15T19:00:53.993 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-01-15T19:00:54.182 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-01-15T19:00:54.247 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-01-15T19:00:54.439 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-01-15T19:00:54.472 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-01-15T19:00:54.662 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-01-15T19:00:54.708 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-01-15T19:00:54.898 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-01-15T19:00:54.930 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-01-15T19:00:55.142 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-01-15T19:00:55.145 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-01-15T19:00:55.363 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-01-15T19:00:55.365 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-01-15T19:00:55.391 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-01-15T19:00:55.594 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-01-15T19:00:55.627 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-01-15T19:00:55.825 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-01-15T19:00:55.857 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-01-15T19:00:56.058 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-01-15T19:00:56.090 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-01-15T19:00:56.263 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-01-15T19:00:56.306 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-01-15T19:00:56.494 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-01-15T19:00:56.536 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-01-15T19:00:56.702 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-01-15T19:00:56.766 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-01-15T19:00:56.904 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-01-15T19:00:56.988 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-01-15T19:00:57.116 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-01-15T19:00:57.211 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-01-15T19:00:57.326 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-01-15T19:00:57.430 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-01-15T19:00:57.534 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-01-15T19:00:57.649 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-01-15T19:00:57.651 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-01-15T19:00:57.743 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-01-15T19:00:57.868 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-01-15T19:00:57.952 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-01-15T19:00:58.099 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-01-15T19:00:58.163 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-01-15T19:00:58.320 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-01-15T19:00:58.372 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-01-15T19:00:58.540 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-01-15T19:00:58.583 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-01-15T19:00:58.760 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-01-15T19:00:58.792 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-01-15T19:00:58.981 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-01-15T19:00:59.003 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-01-15T19:00:59.084 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:00:59 +0000 (0:00:28.471) 0:01:09.468 ******** 2021-01-15T19:00:59.131 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:00:59.133 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.143 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.155 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.156 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.166 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.177 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.179 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.189 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.210 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:00:59.212 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-01-15T19:00:59.397 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:00:59.399 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-01-15T19:00:59.409 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.419 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.421 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-01-15T19:00:59.628 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:00:59.629 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-01-15T19:00:59.639 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:00:59.640 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-01-15T19:00:59.837 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:00:59.839 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-01-15T19:00:59.849 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.859 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.861 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.872 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.884 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.885 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.896 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.908 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.908 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.920 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.931 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.932 INFO:teuthology.task.ansible.out: skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.942 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.953 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.955 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.975 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.977 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.987 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:00:59.998 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.000 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.010 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.021 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.033 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.034 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.044 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.045 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.055 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.066 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.068 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.078 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:01:00.079 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.090 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.093 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.102 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.113 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.125 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.126 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.137 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.148 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.150 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.159 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.171 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.173 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.182 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.183 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.195 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.207 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.218 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.219 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.230 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.232 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.242 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.252 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.254 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.264 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.274 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.276 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.286 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.298 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.309 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:01:00.311 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.332 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.343 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.364 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.375 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.377 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.387 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.399 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.400 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.412 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.423 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.434 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.436 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.446 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.457 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.458 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.468 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.480 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.481 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.491 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.502 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.504 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.526 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.528 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52 2021-01-15T19:01:00.530 INFO:teuthology.task.ansible.out:"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.539 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.541 INFO:teuthology.task.ansible.out: skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.563 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.565 INFO:teuthology.task.ansible.out: skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.575 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.577 INFO:teuthology.task.ansible.out: skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.587 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.598 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.609 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.611 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.621 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.632 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.633 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.643 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.654 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.656 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.666 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.667 INFO:teuthology.task.ansible.out: 2021-01-15T19:01:00.677 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.689 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.699 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.710 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.712 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.722 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.733 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.734 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.744 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:01:00.745 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.755 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.766 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.777 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.779 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.788 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.790 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.799 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.811 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.812 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.822 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.823 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.834 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.844 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.855 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.906 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:00 +0000 (0:00:01.822) 0:01:11.291 ******** 2021-01-15T19:01:00.932 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2021-01-15T19:01:00.934 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:00.966 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:00 +0000 (0:00:00.060) 0:01:11.351 ******** 2021-01-15T19:01:00.972 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:01:00.973 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-01-15T19:01:01.202 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:01.204 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-01-15T19:01:01.224 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:01:01.226 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-01-15T19:01:01.392 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:01.394 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-01-15T19:01:01.446 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-01-15T19:01:01.448 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com", "key_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-01-15T19:01:01.556 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:01 +0000 (0:00:00.590) 0:01:11.941 ******** 2021-01-15T19:01:01.575 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:01.576 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-01-15T19:01:01.596 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:01.598 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.607 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.618 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.620 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.630 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.641 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.652 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.653 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.663 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.770 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:01.771 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-01-15T19:01:01.889 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:01.890 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-01-15T19:01:01.901 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.903 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:01.965 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:01.967 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-01-15T19:01:02.116 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:02.118 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-01-15T19:01:02.148 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:02.150 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-01-15T19:01:02.341 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:02.343 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} TASK [users : Remove revoked users] ******************************************** ok: [smithi136.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "c 2021-01-15T19:01:02.345 INFO:teuthology.task.ansible.out:hanged": false, "item": "shylesh", "name": "shylesh", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.356 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.368 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.380 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.391 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.403 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.414 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.426 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.448 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.460 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.471 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.483 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.494 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.506 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.508 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.518 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.530 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.532 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi136.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.543 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.553 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.554 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.565 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.576 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.587 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.598 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.599 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.609 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.611 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.620 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.631 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.642 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.644 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.653 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.664 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.675 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.676 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.686 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.697 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.708 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.710 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.719 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.721 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi136.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-01-15T19:01:02.731 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:02.733 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.743 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.754 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.756 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.765 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.776 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.787 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.789 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.799 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.801 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.810 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.821 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.834 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.843 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.854 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.856 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.865 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.876 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.887 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.899 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.909 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.911 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi136.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-01-15T19:01:02.920 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:02.922 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.931 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.934 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.943 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.954 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.965 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.966 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.976 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.987 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.997 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:02.999 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.009 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.020 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.030 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.032 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.041 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.052 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.054 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.063 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.074 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.085 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.087 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.096 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:03.098 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.112 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.123 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.124 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.145 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.146 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.158 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.169 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.181 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.182 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.193 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.205 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.217 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.219 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.231 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.241 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.243 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.254 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.257 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.266 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.278 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.281 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.291 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:03.293 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-01-15T19:01:03.293 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-01-15T19:01:03.293 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.305 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.317 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.328 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.329 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.341 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.353 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.364 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.366 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.376 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.387 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.398 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.410 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.412 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.469 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:03 +0000 (0:00:01.912) 0:01:13.854 ******** 2021-01-15T19:01:03.487 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:03.488 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-01-15T19:01:03.499 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2021-01-15T19:01:03.501 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:03.536 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:03 +0000 (0:00:00.067) 0:01:13.921 ******** 2021-01-15T19:01:03.662 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi136.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-01-15T19:01:03.772 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-01-15T19:01:03.847 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-01-15T19:01:03.974 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-01-15T19:01:04.037 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-01-15T19:01:04.172 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-01-15T19:01:04.225 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-01-15T19:01:04.374 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-01-15T19:01:04.418 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-01-15T19:01:04.574 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-01-15T19:01:04.617 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-01-15T19:01:04.779 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-01-15T19:01:04.802 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-01-15T19:01:04.973 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-01-15T19:01:05.005 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-01-15T19:01:05.171 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-01-15T19:01:05.205 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-01-15T19:01:05.365 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-01-15T19:01:05.397 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-01-15T19:01:05.557 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-01-15T19:01:05.591 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-01-15T19:01:05.762 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-01-15T19:01:05.772 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-01-15T19:01:05.973 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-01-15T19:01:05.974 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-01-15T19:01:06.162 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-01-15T19:01:06.174 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-01-15T19:01:06.354 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-01-15T19:01:06.366 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-01-15T19:01:06.554 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-01-15T19:01:06.565 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-01-15T19:01:06.745 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-01-15T19:01:06.756 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-01-15T19:01:06.758 INFO:teuthology.task.ansible.out: 2021-01-15T19:01:06.935 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-01-15T19:01:06.969 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-01-15T19:01:07.130 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-01-15T19:01:07.163 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-01-15T19:01:07.311 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-01-15T19:01:07.365 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-01-15T19:01:07.503 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-01-15T19:01:07.568 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-01-15T19:01:07.686 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-01-15T19:01:07.775 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-01-15T19:01:07.882 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-01-15T19:01:07.979 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-01-15T19:01:08.064 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-01-15T19:01:08.181 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-01-15T19:01:08.392 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-01-15T19:01:08.580 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-01-15T19:01:08.781 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-01-15T19:01:08.978 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-01-15T19:01:09.170 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-01-15T19:01:09.276 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "dorinda", "name": "dorinda", "remove": false, "state": "absent"} 2021-01-15T19:01:09.382 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-01-15T19:01:09.467 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-01-15T19:01:09.582 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-01-15T19:01:09.656 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-01-15T19:01:09.772 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-01-15T19:01:09.836 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-01-15T19:01:09.974 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-01-15T19:01:10.027 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-01-15T19:01:10.165 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-01-15T19:01:10.208 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-01-15T19:01:10.355 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-01-15T19:01:10.408 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-01-15T19:01:10.556 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-01-15T19:01:10.600 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-01-15T19:01:10.757 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-01-15T19:01:10.779 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-01-15T19:01:10.949 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-01-15T19:01:10.971 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-01-15T19:01:11.149 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-01-15T19:01:11.216 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:11 +0000 (0:00:07.680) 0:01:21.601 ******** 2021-01-15T19:01:11.733 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-01-15T19:01:11.734 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "checksum": "e880c991f1562524752e643d227b8575ac6e2ed1", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "fd5444fc24d885aa1caaed5c7ef528f8", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737271.249525-46672739845640/source", "state": "file", "uid": 0} 2021-01-15T19:01:11.767 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:11 +0000 (0:00:00.550) 0:01:22.152 ******** 2021-01-15T19:01:11.796 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-01-15T19:01:11.797 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => { "msg": "Host smithi136.front.sepia.ceph.com is running CentOS 8.2 (Core)" } 2021-01-15T19:01:11.829 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:11 +0000 (0:00:00.062) 0:01:22.214 ******** 2021-01-15T19:01:11.868 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-01-15T19:01:11.869 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2021-01-15T19:01:11.901 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:11 +0000 (0:00:00.071) 0:01:22.286 ******** 2021-01-15T19:01:11.928 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:11.961 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:11 +0000 (0:00:00.059) 0:01:22.346 ******** 2021-01-15T19:01:12.278 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:12.280 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "dorinda", "name": "dorinda", "remove": false, "state": "absent"} 2021-01-15T19:01:12.476 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.320055", "end": "2021-01-15 19:01:12.453050", "rc": 0, "start": "2021-01-15 19:01:12.132995", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-01-15T19:01:12.508 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.546) 0:01:22.893 ******** 2021-01-15T19:01:12.513 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:12.515 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-01-15T19:01:12.536 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-01-15T19:01:12.538 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:12.581 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.073) 0:01:22.966 ******** 2021-01-15T19:01:12.609 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:12.643 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.061) 0:01:23.028 ******** 2021-01-15T19:01:12.672 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-01-15T19:01:12.674 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:12.710 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.066) 0:01:23.094 ******** 2021-01-15T19:01:12.715 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:12.717 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-01-15T19:01:12.738 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:12.772 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.062) 0:01:23.157 ******** 2021-01-15T19:01:12.799 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-01-15T19:01:12.801 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:12.834 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.062) 0:01:23.219 ******** 2021-01-15T19:01:12.872 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-01-15T19:01:12.874 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:12.907 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.072) 0:01:23.292 ******** 2021-01-15T19:01:12.914 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:12.916 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-01-15T19:01:12.947 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-01-15T19:01:12.949 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi136.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:12.981 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:12 +0000 (0:00:00.074) 0:01:23.366 ******** 2021-01-15T19:01:13.013 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-01-15T19:01:13.015 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.046 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.065) 0:01:23.431 ******** 2021-01-15T19:01:13.072 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-01-15T19:01:13.074 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.107 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.061) 0:01:23.492 ******** 2021-01-15T19:01:13.114 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:13.116 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-01-15T19:01:13.137 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi136.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:13.175 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.067) 0:01:23.560 ******** 2021-01-15T19:01:13.215 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-01-15T19:01:13.217 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.264 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.088) 0:01:23.648 ******** 2021-01-15T19:01:13.281 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:13.282 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-01-15T19:01:13.292 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-01-15T19:01:13.294 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:13.326 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.062) 0:01:23.711 ******** 2021-01-15T19:01:13.353 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-01-15T19:01:13.354 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.400 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.073) 0:01:23.785 ******** 2021-01-15T19:01:13.427 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-01-15T19:01:13.429 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.462 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.061) 0:01:23.846 ******** 2021-01-15T19:01:13.478 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:13.480 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-01-15T19:01:13.500 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-01-15T19:01:13.502 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.534 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.072) 0:01:23.919 ******** 2021-01-15T19:01:13.561 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-01-15T19:01:13.563 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.597 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.063) 0:01:23.982 ******** 2021-01-15T19:01:13.624 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-01-15T19:01:13.627 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:13.659 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.062) 0:01:24.044 ******** 2021-01-15T19:01:13.677 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:13.679 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-01-15T19:01:13.689 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.724 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.064) 0:01:24.109 ******** 2021-01-15T19:01:13.751 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-01-15T19:01:13.753 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.785 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.061) 0:01:24.170 ******** 2021-01-15T19:01:13.812 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-01-15T19:01:13.814 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.847 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.061) 0:01:24.232 ******** 2021-01-15T19:01:13.875 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:13.877 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-01-15T19:01:13.887 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:13.921 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:13 +0000 (0:00:00.074) 0:01:24.306 ******** 2021-01-15T19:01:13.959 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-01-15T19:01:13.961 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:14.014 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.092) 0:01:24.399 ******** 2021-01-15T19:01:14.044 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-01-15T19:01:14.046 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:14.091 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.077) 0:01:24.476 ******** 2021-01-15T19:01:14.098 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:14.100 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-01-15T19:01:14.121 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi136.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:14.154 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.063) 0:01:24.539 ******** 2021-01-15T19:01:14.181 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-01-15T19:01:14.182 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:14.218 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.063) 0:01:24.602 ******** 2021-01-15T19:01:14.245 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-01-15T19:01:14.246 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:14.302 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.083) 0:01:24.686 ******** 2021-01-15T19:01:14.311 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2021-01-15T19:01:14.313 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-01-15T19:01:14.373 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.071) 0:01:24.758 ******** 2021-01-15T19:01:14.526 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-01-15T19:01:14.528 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T19:01:14.560 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.186) 0:01:24.945 ******** 2021-01-15T19:01:14.904 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-01-15T19:01:14.906 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "checksum": "e880c991f1562524752e643d227b8575ac6e2ed1", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "fd5444fc24d885aa1caaed5c7ef528f8", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737274.395109-174408597255833/source", "state": "file", "uid": 0} 2021-01-15T19:01:14.943 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:14 +0000 (0:00:00.382) 0:01:25.328 ******** 2021-01-15T19:01:14.980 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-01-15T19:01:14.982 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => { "msg": "Host smithi001.front.sepia.ceph.com is running CentOS 8.2 (Core)" } 2021-01-15T19:01:15.017 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.073) 0:01:25.402 ******** 2021-01-15T19:01:15.056 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-01-15T19:01:15.057 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2021-01-15T19:01:15.094 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.076) 0:01:25.479 ******** 2021-01-15T19:01:15.121 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-01-15T19:01:15.123 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:15.156 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.062) 0:01:25.541 ******** 2021-01-15T19:01:15.320 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-01-15T19:01:15.321 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2021-01-15T19:01:15.682 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-01-15T19:01:15.684 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.314584", "end": "2021-01-15 19:01:15.655666", "rc": 0, "start": "2021-01-15 19:01:15.341082", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-01-15T19:01:15.716 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.560) 0:01:26.101 ******** 2021-01-15T19:01:15.743 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-01-15T19:01:15.745 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi136.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2021-01-15T19:01:15.781 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.064) 0:01:26.166 ******** 2021-01-15T19:01:15.812 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.030) 0:01:26.197 ******** 2021-01-15T19:01:15.818 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-01-15T19:01:15.820 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:15.855 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.042) 0:01:26.240 ******** 2021-01-15T19:01:15.882 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-01-15T19:01:15.884 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:15.917 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.062) 0:01:26.302 ******** 2021-01-15T19:01:15.943 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-01-15T19:01:15.945 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:15.982 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:15 +0000 (0:00:00.065) 0:01:26.367 ******** 2021-01-15T19:01:16.010 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-01-15T19:01:16.011 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.053 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.070) 0:01:26.438 ******** 2021-01-15T19:01:16.081 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-01-15T19:01:16.082 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.118 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.065) 0:01:26.503 ******** 2021-01-15T19:01:16.155 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-01-15T19:01:16.157 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:16.187 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.069) 0:01:26.572 ******** 2021-01-15T19:01:16.214 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-01-15T19:01:16.216 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.246 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.058) 0:01:26.631 ******** 2021-01-15T19:01:16.272 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-01-15T19:01:16.274 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi136.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2021-01-15T19:01:16.307 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.060) 0:01:26.692 ******** 2021-01-15T19:01:16.334 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-01-15T19:01:16.336 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:16.367 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.059) 0:01:26.751 ******** 2021-01-15T19:01:16.392 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-01-15T19:01:16.394 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.424 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.057) 0:01:26.809 ******** 2021-01-15T19:01:16.452 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-01-15T19:01:16.453 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:16.484 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.059) 0:01:26.869 ******** 2021-01-15T19:01:16.521 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-01-15T19:01:16.638 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 15 January 2021 19:01:16 +0000 (0:00:00.075) 0:01:26.944 ******** TASK [common : List CDN releases available to system] ************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 15 January 2021 19:01:16 +0000 (0:00:00.070) 0:01:27.015 ******** 2021-01-15T19:01:16.657 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-01-15T19:01:16.659 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.688 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.057) 0:01:27.073 ******** 2021-01-15T19:01:16.714 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-01-15T19:01:16.715 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-01-15T19:01:16.716 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.749 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.061) 0:01:27.134 ******** 2021-01-15T19:01:16.780 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.031) 0:01:27.165 ******** 2021-01-15T19:01:16.786 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-01-15T19:01:16.788 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:16.822 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.041) 0:01:27.207 ******** 2021-01-15T19:01:16.828 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-01-15T19:01:16.830 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.863 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.041) 0:01:27.248 ******** 2021-01-15T19:01:16.869 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-01-15T19:01:16.871 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.904 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.040) 0:01:27.289 ******** 2021-01-15T19:01:16.911 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-01-15T19:01:16.913 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.945 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.041) 0:01:27.330 ******** 2021-01-15T19:01:16.951 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-01-15T19:01:16.952 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:16.984 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:16 +0000 (0:00:00.038) 0:01:27.369 ******** 2021-01-15T19:01:16.989 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-01-15T19:01:16.991 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.026 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.038) 0:01:27.407 ******** 2021-01-15T19:01:17.028 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-01-15T19:01:17.029 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.063 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.040) 0:01:27.448 ******** 2021-01-15T19:01:17.072 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-01-15T19:01:17.074 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.114 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.051) 0:01:27.499 ******** 2021-01-15T19:01:17.120 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-01-15T19:01:17.122 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.157 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.042) 0:01:27.541 ******** 2021-01-15T19:01:17.163 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-01-15T19:01:17.165 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.198 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.041) 0:01:27.583 ******** 2021-01-15T19:01:17.205 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-01-15T19:01:17.207 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.245 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.046) 0:01:27.630 ******** 2021-01-15T19:01:17.251 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-01-15T19:01:17.253 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.290 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.044) 0:01:27.674 ******** 2021-01-15T19:01:17.296 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-01-15T19:01:17.297 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.332 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.041) 0:01:27.716 ******** 2021-01-15T19:01:17.337 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-01-15T19:01:17.339 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.377 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.045) 0:01:27.761 ******** 2021-01-15T19:01:17.382 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-01-15T19:01:17.384 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-01-15T19:01:17.417 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.039) 0:01:27.801 ******** 2021-01-15T19:01:17.423 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-01-15T19:01:17.424 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.461 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.044) 0:01:27.846 ******** 2021-01-15T19:01:17.467 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-01-15T19:01:17.469 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.504 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.042) 0:01:27.888 ******** 2021-01-15T19:01:17.510 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-01-15T19:01:17.511 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.547 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.043) 0:01:27.931 ******** 2021-01-15T19:01:17.553 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-01-15T19:01:17.555 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:17.588 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.041) 0:01:27.973 ******** 2021-01-15T19:01:17.823 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-01-15T19:01:17.825 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T19:01:17.857 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:17 +0000 (0:00:00.268) 0:01:28.242 ******** 2021-01-15T19:01:18.385 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-01-15T19:01:18.387 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2021-01-15T19:01:18.521 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-01-15T19:01:18.523 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.782790", "end": "2021-01-15 19:01:18.500335", "rc": 0, "start": "2021-01-15 19:01:17.717545", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:01:18.557 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:18 +0000 (0:00:00.699) 0:01:28.941 ******** 2021-01-15T19:01:18.847 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-01-15T19:01:18.848 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2021-01-15T19:01:18.883 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:18 +0000 (0:00:00.326) 0:01:29.268 ******** 2021-01-15T19:01:19.370 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi001.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2021-01-15T19:01:19.819 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2021-01-15T19:01:19.856 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:19 +0000 (0:00:00.972) 0:01:30.241 ******** 2021-01-15T19:01:19.883 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-01-15T19:01:19.885 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:19.918 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:19 +0000 (0:00:00.061) 0:01:30.302 ******** 2021-01-15T19:01:19.944 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:19.980 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:19 +0000 (0:00:00.062) 0:01:30.365 ******** 2021-01-15T19:01:20.008 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-01-15T19:01:20.010 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.041 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.061) 0:01:30.426 ******** 2021-01-15T19:01:20.068 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.102 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.060) 0:01:30.487 ******** 2021-01-15T19:01:20.129 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.164 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.062) 0:01:30.549 ******** 2021-01-15T19:01:20.191 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-01-15T19:01:20.192 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.224 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.059) 0:01:30.609 ******** 2021-01-15T19:01:20.250 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-01-15T19:01:20.252 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.284 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.059) 0:01:30.669 ******** 2021-01-15T19:01:20.312 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-01-15T19:01:20.314 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.346 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.062) 0:01:30.731 ******** 2021-01-15T19:01:20.374 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-01-15T19:01:20.377 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:20.407 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:20 +0000 (0:00:00.060) 0:01:30.792 ******** 2021-01-15T19:01:22.532 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-01-15T19:01:22.534 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.859961", "end": "2021-01-15 19:01:22.501726", "rc": 0, "start": "2021-01-15 19:01:20.641765", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:01:22.567 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:22 +0000 (0:00:02.160) 0:01:32.952 ******** 2021-01-15T19:01:25.648 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-01-15T19:01:25.650 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: krb5-workstation", "Installed: krb5-workstation-1.18.2-5.el8.x86_64", "Installed: krb5-libs-1.18.2-5.el8.x86_64", "Installed: libkadm5-1.18.2-5.el8.x86_64", "Removed: krb5-workstation-1.17-18.el8.x86_64", "Removed: libkadm5-1.17-18.el8.x86_64", "Removed: krb5-libs-1.17-18.el8.x86_64"]} 2021-01-15T19:01:25.684 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:25 +0000 (0:00:03.117) 0:01:36.069 ******** 2021-01-15T19:01:25.711 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-01-15T19:01:25.713 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:25.745 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:25 +0000 (0:00:00.060) 0:01:36.130 ******** 2021-01-15T19:01:25.772 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-01-15T19:01:25.773 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:25.808 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:25 +0000 (0:00:00.062) 0:01:36.193 ******** 2021-01-15T19:01:25.835 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-01-15T19:01:25.837 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:25.870 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:25 +0000 (0:00:00.061) 0:01:36.255 ******** 2021-01-15T19:01:25.897 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-01-15T19:01:25.899 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:25.937 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:25 +0000 (0:00:00.067) 0:01:36.322 ******** 2021-01-15T19:01:26.443 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-01-15T19:01:26.445 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2021-01-15T19:01:26.490 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:26 +0000 (0:00:00.553) 0:01:36.875 ******** 2021-01-15T19:01:29.419 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-01-15T19:01:29.421 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2021-01-15T19:01:29.453 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:29 +0000 (0:00:02.962) 0:01:39.838 ******** 2021-01-15T19:01:29.905 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-01-15T19:01:29.907 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: krb5-workstation", "Installed: krb5-workstation-1.18.2-5.el8.x86_64", "Installed: krb5-libs-1.18.2-5.el8.x86_64", "Installed: libkadm5-1.18.2-5.el8.x86_64", "Removed: krb5-workstation-1.17-18.el8.x86_64", "Removed: libkadm5-1.17-18.el8.x86_64", "Removed: krb5-libs-1.17-18.el8.x86_64"]} 2021-01-15T19:01:29.940 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:29 +0000 (0:00:00.487) 0:01:40.325 ******** 2021-01-15T19:01:29.947 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-15T19:01:29.949 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2021-01-15T19:01:29.969 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-01-15T19:01:29.972 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:30.003 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:30 +0000 (0:00:00.062) 0:01:40.388 ******** 2021-01-15T19:01:30.030 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-01-15T19:01:30.032 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:30.064 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:30 +0000 (0:00:00.060) 0:01:40.449 ******** 2021-01-15T19:01:30.091 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-01-15T19:01:30.093 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:30.125 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:30 +0000 (0:00:00.061) 0:01:40.510 ******** 2021-01-15T19:01:30.152 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-01-15T19:01:30.153 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:30.186 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:30 +0000 (0:00:00.060) 0:01:40.571 ******** 2021-01-15T19:01:30.359 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-15T19:01:30.361 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2021-01-15T19:01:30.662 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-01-15T19:01:30.664 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2021-01-15T19:01:30.696 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:30 +0000 (0:00:00.510) 0:01:41.081 ******** 2021-01-15T19:01:30.787 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-15T19:01:30.788 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2021-01-15T19:01:30.824 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:30 +0000 (0:00:00.127) 0:01:41.209 ******** 2021-01-15T19:01:31.016 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-01-15T19:01:31.018 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:01:31.054 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:31 +0000 (0:00:00.230) 0:01:41.439 ******** 2021-01-15T19:01:31.568 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-01-15T19:01:31.569 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737291.0766642-127224908304140/source", "state": "file", "uid": 0} 2021-01-15T19:01:31.972 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2021-01-15T19:01:32.401 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2021-01-15T19:01:32.438 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:32 +0000 (0:00:01.384) 0:01:42.823 ******** 2021-01-15T19:01:32.466 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-01-15T19:01:32.468 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:32.500 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:32 +0000 (0:00:00.061) 0:01:42.885 ******** 2021-01-15T19:01:32.537 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-01-15T19:01:32.539 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2021-01-15T19:01:32.571 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:32 +0000 (0:00:00.070) 0:01:42.956 ******** 2021-01-15T19:01:32.764 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-01-15T19:01:32.766 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-01-15T19:01:32.798 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:32 +0000 (0:00:00.227) 0:01:43.183 ******** 2021-01-15T19:01:32.836 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-01-15T19:01:32.838 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:32.847 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:32.857 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:32.868 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:32.901 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:32 +0000 (0:00:00.102) 0:01:43.286 ******** 2021-01-15T19:01:33.891 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-01-15T19:01:33.892 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: smartmontools", "Installed: smartmontools-1:7.1-1.el8.x86_64", "Removed: smartmontools-1:6.6-3.el8.x86_64"]} 2021-01-15T19:01:33.927 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:33 +0000 (0:00:01.025) 0:01:44.311 ******** 2021-01-15T19:01:33.933 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-01-15T19:01:33.935 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2021-01-15T19:01:33.970 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:33 +0000 (0:00:00.042) 0:01:44.354 ******** 2021-01-15T19:01:33.996 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-01-15T19:01:33.998 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:34.036 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:34 +0000 (0:00:00.066) 0:01:44.421 ******** 2021-01-15T19:01:34.426 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-15T19:01:34.428 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2021-01-15T19:01:34.479 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-01-15T19:01:34.481 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2021-01-15T19:01:34.512 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:34 +0000 (0:00:00.476) 0:01:44.897 ******** 2021-01-15T19:01:34.881 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-15T19:01:34.883 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2021-01-15T19:01:34.955 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-01-15T19:01:34.957 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2021-01-15T19:01:34.988 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:34 +0000 (0:00:00.475) 0:01:45.373 ******** 2021-01-15T19:01:35.025 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-01-15T19:01:35.027 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:35.059 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:35 +0000 (0:00:00.071) 0:01:45.444 ******** 2021-01-15T19:01:35.252 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-01-15T19:01:35.254 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T19:01:35.289 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:35 +0000 (0:00:00.229) 0:01:45.674 ******** 2021-01-15T19:01:35.305 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-01-15T19:01:35.307 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2021-01-15T19:01:35.343 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:35 +0000 (0:00:00.053) 0:01:45.728 ******** 2021-01-15T19:01:35.506 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-01-15T19:01:35.507 INFO:teuthology.task.ansible.out:fatal: [smithi136.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2021-01-15T19:01:35.551 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:35 +0000 (0:00:00.207) 0:01:45.936 ******** 2021-01-15T19:01:35.557 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-01-15T19:01:35.559 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:01:35.598 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:35 +0000 (0:00:00.047) 0:01:45.983 ******** 2021-01-15T19:01:35.604 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-01-15T19:01:35.605 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:35.641 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:35 +0000 (0:00:00.042) 0:01:46.025 ******** 2021-01-15T19:01:36.098 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-01-15T19:01:36.100 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2021-01-15T19:01:36.132 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:36 +0000 (0:00:00.491) 0:01:46.517 ******** 2021-01-15T19:01:36.139 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-01-15T19:01:36.141 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": true, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "md5sum": "14041bf2666d8f8d277567958c7e9f69", "mode": "0755", "owner": "root", "secontext": "system_u:object_r:bin_t:s0", "size": 13355, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737295.6190722-106588923048078/source", "state": "file", "uid": 0} 2021-01-15T19:01:36.570 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2021-01-15T19:01:36.873 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-01-15T19:01:36.874 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2021-01-15 18:59:33 UTC", "ActiveEnterTimestampMonotonic": "13246813", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target network.target local-fs.target tmp.mount time-sync.target basic.target systemd-tmpfiles-setup.service sysinit.target -.mount system.slice var-run.mount systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2021-01-15 18:59:33 UTC", "AssertTimestampMonotonic": "13245440", "Before": "shutdown.target xdm.service multi-user.target plymouth-quit.service getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2021-01-15 18:59:33 UTC", "ConditionTimestampMonotonic": "13245440", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "Co 2021-01-15T19:01:36.876 INFO:teuthology.task.ansible.out:ntrolGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "940", "ExecMainStartTimestamp": "Fri 2021-01-15 18:59:33 UTC", "ExecMainStartTimestampMonotonic": "13246742", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "984", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2021-01-15 18:59:33 UTC", "InactiveExitTimestampMonotonic": "13246813", "InvocationID": "a295675ae8ca4c57960fb66598db9f3e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", " 2021-01-15T19:01:36.877 INFO:teuthology.task.ansible.out:LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127261", "LimitNPROCSoft": "127261", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127261", "LimitSIGPENDINGSoft": "127261", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "940", "MemoryAccounting": "yes", "MemoryCurrent": "1744896", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /run/nrpe", 2021-01-15T19:01:36.879 INFO:teuthology.task.ansible.out:"Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2021-01-15 18:59:33 UTC", "StateChangeTimestampMonotonic": "13246813", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203618", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "988", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2021-01-15 18:59:33 UTC", "WatchdogTimestampMonotonic": "13246812", "WatchdogUSec": "0"}} 2021-01-15T19:01:36.908 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:36 +0000 (0:00:00.776) 0:01:47.293 ******** 2021-01-15T19:01:36.997 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-01-15T19:01:36.999 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2021-01-15T19:01:37.036 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:37 +0000 (0:00:00.128) 0:01:47.421 ******** 2021-01-15T19:01:37.063 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-01-15T19:01:37.065 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:37.097 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:37 +0000 (0:00:00.060) 0:01:47.482 ******** 2021-01-15T19:01:37.102 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-01-15T19:01:37.104 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001765", "end": "2021-01-15 19:01:37.074469", "rc": 0, "start": "2021-01-15 19:01:37.072704", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2021-01-15T19:01:37.137 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:37 +0000 (0:00:00.040) 0:01:47.522 ******** 2021-01-15T19:01:37.143 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-01-15T19:01:37.145 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2021-01-15T19:01:37.179 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:37 +0000 (0:00:00.041) 0:01:47.564 ******** 2021-01-15T19:01:37.383 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-01-15T19:01:37.385 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-01-15T19:01:37.418 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:37 +0000 (0:00:00.239) 0:01:47.803 ******** 2021-01-15T19:01:37.456 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-01-15T19:01:37.457 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:37.467 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:37.478 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:37.489 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:37.524 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:37 +0000 (0:00:00.105) 0:01:47.909 ******** 2021-01-15T19:01:38.590 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-01-15T19:01:38.591 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load"]} 2021-01-15T19:01:38.623 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:38 +0000 (0:00:01.099) 0:01:49.008 ******** 2021-01-15T19:01:38.649 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-01-15T19:01:38.652 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:38.682 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:38 +0000 (0:00:00.058) 0:01:49.067 ******** 2021-01-15T19:01:39.160 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-01-15T19:01:39.161 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2021-01-15T19:01:39.198 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:39 +0000 (0:00:00.515) 0:01:49.583 ******** 2021-01-15T19:01:39.661 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-01-15T19:01:39.663 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2021-01-15T19:01:39.695 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:39 +0000 (0:00:00.496) 0:01:50.080 ******** 2021-01-15T19:01:39.732 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-01-15T19:01:39.734 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:39.744 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2021-01-15T19:01:39.781 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:39 +0000 (0:00:00.086) 0:01:50.166 ******** 2021-01-15T19:01:39.831 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-01-15T19:01:39.833 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils", "Installed: python3-libsemanage-2.9-3.el8.x86_64", "Installed: libsemanage-2.9-3.el8.x86_64", "Removed: python3-libsemanage-2.9-2.el8.x86_64", "Removed: libsemanage-2.9-2.el8.x86_64"]} 2021-01-15T19:01:39.869 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:39 +0000 (0:00:00.087) 0:01:50.253 ******** 2021-01-15T19:01:39.995 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T19:01:40.033 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:40 +0000 (0:00:00.164) 0:01:50.418 ******** 2021-01-15T19:01:40.271 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-01-15T19:01:40.272 INFO:teuthology.task.ansible.out:fatal: [smithi001.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2021-01-15T19:01:40.306 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:40 +0000 (0:00:00.273) 0:01:50.691 ******** 2021-01-15T19:01:40.334 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-01-15T19:01:40.336 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:01:40.371 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:40 +0000 (0:00:00.064) 0:01:50.756 ******** 2021-01-15T19:01:40.854 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-01-15T19:01:40.856 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2021-01-15T19:01:40.888 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:40 +0000 (0:00:00.517) 0:01:51.273 ******** 2021-01-15T19:01:41.206 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-01-15T19:01:41.207 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2021-01-15 18:59:23 UTC", "ActiveEnterTimestampMonotonic": "16632541", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "system.slice basic.target time-sync.target sysinit.target tmp.mount -.mount systemd-journald.socket nss-lookup.target local-fs.target systemd-tmpfiles-setup.service var-run.mount network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2021-01-15 18:59:23 UTC", "AssertTimestampMonotonic": "16631471", "Before": "xdm.service getty@tty1.service shutdown.target plymouth-quit.service multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2021-01-15 18:59:23 UTC", "ConditionTimestampMonotonic": "16631470", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "Co 2021-01-15T19:01:41.209 INFO:teuthology.task.ansible.out:ntrolGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "947", "ExecMainStartTimestamp": "Fri 2021-01-15 18:59:23 UTC", "ExecMainStartTimestampMonotonic": "16632514", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "984", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2021-01-15 18:59:23 UTC", "InactiveExitTimestampMonotonic": "16632541", "InvocationID": "f1133205f6fb45508af8fb0916027fcd", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", " 2021-01-15T19:01:41.210 INFO:teuthology.task.ansible.out:LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127286", "LimitNPROCSoft": "127286", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127286", "LimitSIGPENDINGSoft": "127286", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "947", "MemoryAccounting": "yes", "MemoryCurrent": "1720320", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", 2021-01-15T19:01:41.212 INFO:teuthology.task.ansible.out:"Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2021-01-15 18:59:23 UTC", "StateChangeTimestampMonotonic": "16632541", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203657", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "988", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2021-01-15 18:59:23 UTC", "WatchdogTimestampMonotonic": "16632540", "WatchdogUSec": "0"}} 2021-01-15T19:01:41.241 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:41 +0000 (0:00:00.353) 0:01:51.626 ******** 2021-01-15T19:01:41.437 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001815", "end": "2021-01-15 19:01:41.417540", "rc": 0, "start": "2021-01-15 19:01:41.415725", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2021-01-15T19:01:41.469 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:41 +0000 (0:00:00.227) 0:01:51.854 ******** 2021-01-15T19:01:44.766 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-01-15T19:01:44.768 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils", "Installed: python3-libsemanage-2.9-3.el8.x86_64", "Installed: libsemanage-2.9-3.el8.x86_64", "Removed: python3-libsemanage-2.9-2.el8.x86_64", "Removed: libsemanage-2.9-2.el8.x86_64"]} 2021-01-15T19:01:44.823 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:01:44 +0000 (0:00:03.354) 0:01:55.208 ******** 2021-01-15T19:02:18.168 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-01-15T19:02:18.170 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: selinux-policy-targeted", "Installed: selinux-policy-targeted-3.14.3-54.el8.noarch", "Installed: selinux-policy-3.14.3-54.el8.noarch", "Removed: selinux-policy-3.14.3-41.el8_2.8.noarch", "Removed: selinux-policy-targeted-3.14.3-41.el8_2.8.noarch"]} 2021-01-15T19:02:18.204 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:18 +0000 (0:00:33.381) 0:02:28.589 ******** 2021-01-15T19:02:18.918 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2021-01-15T19:02:18.951 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:18 +0000 (0:00:00.746) 0:02:29.336 ******** 2021-01-15T19:02:24.927 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-01-15T19:02:24.929 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: selinux-policy-targeted", "Installed: selinux-policy-targeted-3.14.3-54.el8.noarch", "Installed: selinux-policy-3.14.3-54.el8.noarch", "Removed: selinux-policy-3.14.3-41.el8_2.8.noarch", "Removed: selinux-policy-targeted-3.14.3-41.el8_2.8.noarch"]} 2021-01-15T19:02:24.964 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:24 +0000 (0:00:06.012) 0:02:35.349 ******** 2021-01-15T19:02:25.430 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-01-15T19:02:25.432 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2021-01-15T19:02:25.466 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:25 +0000 (0:00:00.501) 0:02:35.851 ******** 2021-01-15T19:02:35.646 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-01-15T19:02:35.649 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.509539", "end": "2021-01-15 19:02:35.624833", "failed_when_result": false, "rc": 0, "start": "2021-01-15 19:02:19.115294", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2021-01-15T19:02:35.686 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:35 +0000 (0:00:10.219) 0:02:46.071 ******** 2021-01-15T19:02:36.142 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2021-01-15T19:02:36.177 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:36 +0000 (0:00:00.490) 0:02:46.561 ******** 2021-01-15T19:02:36.372 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-01-15T19:02:36.374 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.009505", "end": "2021-01-15 19:02:36.354616", "rc": 0, "start": "2021-01-15 19:02:36.345111", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:02:36.417 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:36 +0000 (0:00:00.240) 0:02:46.802 ******** 2021-01-15T19:02:36.621 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-01-15T19:02:36.623 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.006200", "end": "2021-01-15 19:02:36.594440", "rc": 0, "start": "2021-01-15 19:02:36.588240", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:02:36.656 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:36 +0000 (0:00:00.238) 0:02:47.040 ******** 2021-01-15T19:02:42.787 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-01-15T19:02:42.789 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:17.118787", "end": "2021-01-15 19:02:42.758332", "failed_when_result": false, "rc": 0, "start": "2021-01-15 19:02:25.639545", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2021-01-15T19:02:42.823 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:42 +0000 (0:00:06.167) 0:02:53.207 ******** 2021-01-15T19:02:43.284 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-01-15T19:02:43.286 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2021-01-15T19:02:43.339 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:43 +0000 (0:00:00.515) 0:02:53.723 ******** 2021-01-15T19:02:43.544 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-01-15T19:02:43.546 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.012388", "end": "2021-01-15 19:02:43.523872", "rc": 0, "start": "2021-01-15 19:02:43.511484", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:02:43.578 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:43 +0000 (0:00:00.239) 0:02:53.963 ******** 2021-01-15T19:02:43.793 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-01-15T19:02:43.795 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012776", "end": "2021-01-15 19:02:43.766996", "rc": 0, "start": "2021-01-15 19:02:43.754220", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:02:43.826 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:43 +0000 (0:00:00.248) 0:02:54.211 ******** 2021-01-15T19:02:53.325 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-01-15T19:02:53.327 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.476852", "end": "2021-01-15 19:02:53.306053", "rc": 0, "start": "2021-01-15 19:02:36.829201", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:02:53.360 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:53 +0000 (0:00:09.533) 0:03:03.745 ******** 2021-01-15T19:02:53.568 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-01-15T19:02:53.570 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2021-01-15T19:02:53.629 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:53 +0000 (0:00:00.268) 0:03:04.013 ******** 2021-01-15T19:02:53.656 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-01-15T19:02:53.659 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2021-01-15T19:02:53.712 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:53 +0000 (0:00:00.083) 0:03:04.097 ******** 2021-01-15T19:02:53.760 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-01-15T19:02:53.762 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-01-15T19:02:53.798 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:53 +0000 (0:00:00.085) 0:03:04.182 ******** 2021-01-15T19:02:53.868 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-01-15T19:02:53.868 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2021-01-15T19:02:53.907 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:53 +0000 (0:00:00.109) 0:03:04.292 ******** 2021-01-15T19:02:53.955 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-01-15T19:02:53.957 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-01-15T19:02:54.006 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:54 +0000 (0:00:00.098) 0:03:04.391 ******** 2021-01-15T19:02:54.213 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-01-15T19:02:54.214 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-01-15T19:02:54.249 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:54 +0000 (0:00:00.242) 0:03:04.634 ******** 2021-01-15T19:02:54.446 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-01-15T19:02:54.447 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2021-01-15T19:02:54.479 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:54 +0000 (0:00:00.230) 0:03:04.864 ******** 2021-01-15T19:02:54.506 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:02:54.539 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:54 +0000 (0:00:00.059) 0:03:04.924 ******** 2021-01-15T19:02:54.565 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-01-15T19:02:54.567 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:02:54.599 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:54 +0000 (0:00:00.059) 0:03:04.984 ******** 2021-01-15T19:02:54.817 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-01-15T19:02:54.818 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2021-01-15T19:02:54.852 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:54 +0000 (0:00:00.252) 0:03:05.236 ******** 2021-01-15T19:02:55.375 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-01-15T19:02:55.377 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2021-01-15T19:02:55.410 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:55 +0000 (0:00:00.558) 0:03:05.795 ******** 2021-01-15T19:02:55.437 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-01-15T19:02:55.438 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi136"}, "changed": false} 2021-01-15T19:02:55.474 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:55 +0000 (0:00:00.063) 0:03:05.859 ******** 2021-01-15T19:02:56.474 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-01-15T19:02:56.476 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi136.front.sepia.ceph.com", "ansible_hostname": "smithi136", "ansible_nodename": "smithi136"}, "changed": true, "name": "smithi136"} 2021-01-15T19:02:56.507 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:56 +0000 (0:00:01.033) 0:03:06.892 ******** 2021-01-15T19:02:56.947 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-01-15T19:02:56.949 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2021-01-15T19:02:56.980 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:56 +0000 (0:00:00.473) 0:03:07.365 ******** 2021-01-15T19:02:57.422 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-01-15T19:02:57.424 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2021-01-15T19:02:57.457 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:57 +0000 (0:00:00.476) 0:03:07.842 ******** 2021-01-15T19:02:58.086 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-01-15T19:02:58.088 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-01-15T19:02:58.120 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:58 +0000 (0:00:00.662) 0:03:08.505 ******** 2021-01-15T19:02:58.580 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-01-15T19:02:58.582 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2021-01-15T19:02:58.614 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:58 +0000 (0:00:00.493) 0:03:08.998 ******** 2021-01-15T19:02:58.806 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-01-15T19:02:58.808 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2021-01-15T19:02:58.841 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:58 +0000 (0:00:00.227) 0:03:09.226 ******** 2021-01-15T19:02:59.033 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-15T19:02:59.035 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2021-01-15T19:02:59.067 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:59 +0000 (0:00:00.226) 0:03:09.452 ******** 2021-01-15T19:02:59.280 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-15T19:02:59.282 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-01-15T19:02:59.314 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:59 +0000 (0:00:00.246) 0:03:09.699 ******** 2021-01-15T19:02:59.766 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2021-01-15T19:02:59.799 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:02:59 +0000 (0:00:00.484) 0:03:10.184 ******** 2021-01-15T19:03:00.263 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-01-15T19:03:00.265 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2021-01-15T19:03:00.296 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:00 +0000 (0:00:00.497) 0:03:10.681 ******** 2021-01-15T19:03:01.174 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-01-15T19:03:01.176 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:17.156422", "end": "2021-01-15 19:03:01.155172", "rc": 0, "start": "2021-01-15 19:02:43.998750", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:03:01.210 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.913) 0:03:11.595 ******** 2021-01-15T19:03:01.414 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-01-15T19:03:01.415 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2021-01-15T19:03:01.447 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.237) 0:03:11.832 ******** 2021-01-15T19:03:01.473 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-01-15T19:03:01.474 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2021-01-15T19:03:01.507 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.059) 0:03:11.892 ******** 2021-01-15T19:03:01.544 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-01-15T19:03:01.546 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-01-15T19:03:01.578 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.071) 0:03:11.963 ******** 2021-01-15T19:03:01.616 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-01-15T19:03:01.618 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2021-01-15T19:03:01.651 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.072) 0:03:12.035 ******** 2021-01-15T19:03:01.687 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-01-15T19:03:01.689 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-01-15T19:03:01.721 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.070) 0:03:12.106 ******** 2021-01-15T19:03:01.914 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-01-15T19:03:01.916 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-01-15T19:03:01.948 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:01 +0000 (0:00:00.226) 0:03:12.333 ******** 2021-01-15T19:03:02.152 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-01-15T19:03:02.154 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2021-01-15T19:03:02.188 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:02 +0000 (0:00:00.240) 0:03:12.573 ******** 2021-01-15T19:03:02.215 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-01-15T19:03:02.217 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:02.248 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:02 +0000 (0:00:00.059) 0:03:12.633 ******** 2021-01-15T19:03:02.278 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-01-15T19:03:02.279 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:02.311 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:02 +0000 (0:00:00.063) 0:03:12.696 ******** 2021-01-15T19:03:02.453 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-01-15T19:03:02.454 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.965474", "end": "2021-01-15 19:03:02.425075", "rc": 0, "start": "2021-01-15 19:03:00.459601", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:03:02.490 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:02 +0000 (0:00:00.178) 0:03:12.874 ******** 2021-01-15T19:03:02.528 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-01-15T19:03:02.530 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2021-01-15T19:03:02.567 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:02 +0000 (0:00:00.077) 0:03:12.952 ******** 2021-01-15T19:03:02.969 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** ok: [smithi136.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2021-01-15T19:03:03.105 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-01-15T19:03:03.107 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2021-01-15T19:03:03.139 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:03 +0000 (0:00:00.572) 0:03:13.524 ******** 2021-01-15T19:03:03.166 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-01-15T19:03:03.168 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi001"}, "changed": false} 2021-01-15T19:03:03.200 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:03 +0000 (0:00:00.060) 0:03:13.585 ******** 2021-01-15T19:03:03.393 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-01-15T19:03:03.395 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2021-01-15T19:03:03.788 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-01-15T19:03:03.790 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi001.front.sepia.ceph.com", "ansible_hostname": "smithi001", "ansible_nodename": "smithi001"}, "changed": true, "name": "smithi001"} 2021-01-15T19:03:03.822 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:03 +0000 (0:00:00.622) 0:03:14.207 ******** 2021-01-15T19:03:03.828 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-01-15T19:03:03.830 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2021-01-15T19:03:03.866 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:03 +0000 (0:00:00.043) 0:03:14.251 ******** 2021-01-15T19:03:04.279 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-01-15T19:03:04.281 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2021-01-15T19:03:04.314 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:04 +0000 (0:00:00.447) 0:03:14.698 ******** 2021-01-15T19:03:04.330 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-01-15T19:03:04.332 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2021-01-15T19:03:04.372 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:04 +0000 (0:00:00.057) 0:03:14.756 ******** 2021-01-15T19:03:04.824 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-01-15T19:03:04.826 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2021-01-15T19:03:04.858 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:04 +0000 (0:00:00.486) 0:03:15.243 ******** 2021-01-15T19:03:04.958 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-01-15T19:03:04.960 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2021-01-15T19:03:05.386 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2021-01-15T19:03:05.521 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-01-15T19:03:05.523 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-01-15T19:03:05.554 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:05 +0000 (0:00:00.695) 0:03:15.939 ******** 2021-01-15T19:03:05.820 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-01-15T19:03:05.822 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2021-01-15T19:03:05.859 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:05 +0000 (0:00:00.305) 0:03:16.244 ******** 2021-01-15T19:03:05.886 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-01-15T19:03:05.922 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:05 +0000 (0:00:00.062) 0:03:16.307 ******** 2021-01-15T19:03:05.948 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-01-15T19:03:05.950 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:05.982 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:05 +0000 (0:00:00.060) 0:03:16.367 ******** 2021-01-15T19:03:06.019 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-01-15T19:03:06.021 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-01-15T19:03:06.056 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.073) 0:03:16.441 ******** 2021-01-15T19:03:06.064 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-01-15T19:03:06.064 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2021-01-15T19:03:06.105 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.049) 0:03:16.490 ******** 2021-01-15T19:03:06.112 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-01-15T19:03:06.113 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:06.147 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.041) 0:03:16.532 ******** 2021-01-15T19:03:06.174 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:03:06.176 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2021-01-15T19:03:06.208 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.060) 0:03:16.592 ******** 2021-01-15T19:03:06.244 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-01-15T19:03:06.245 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:06.278 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.070) 0:03:16.662 ******** 2021-01-15T19:03:06.315 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-01-15T19:03:06.317 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:06.355 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.077) 0:03:16.740 ******** 2021-01-15T19:03:06.391 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.035) 0:03:16.776 ******** 2021-01-15T19:03:06.555 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-15T19:03:06.557 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2021-01-15T19:03:06.594 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.202) 0:03:16.978 ******** 2021-01-15T19:03:06.600 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-01-15T19:03:06.601 INFO:teuthology.task.ansible.out:fatal: [smithi136.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007116", "end": "2021-01-15 19:03:06.570689", "msg": "non-zero return code", "rc": 1, "start": "2021-01-15 19:03:06.563573", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2021-01-15T19:03:06.640 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.046) 0:03:17.025 ******** 2021-01-15T19:03:06.677 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-01-15T19:03:06.679 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:06.715 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.075) 0:03:17.100 ******** 2021-01-15T19:03:06.814 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-15T19:03:06.816 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-01-15T19:03:06.849 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:06 +0000 (0:00:00.133) 0:03:17.234 ******** 2021-01-15T19:03:07.321 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-01-15T19:03:07.322 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2021-01-15T19:03:07.354 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:07 +0000 (0:00:00.504) 0:03:17.739 ******** 2021-01-15T19:03:07.837 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-01-15T19:03:07.838 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2021-01-15T19:03:07.871 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:07 +0000 (0:00:00.516) 0:03:18.256 ******** 2021-01-15T19:03:08.984 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-15T19:03:08.985 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: librbd1-1:12.2.7-9.el8.x86_64", "Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64"]} 2021-01-15T19:03:09.032 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:09 +0000 (0:00:01.161) 0:03:19.417 ******** 2021-01-15T19:03:10.216 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.145406", "end": "2021-01-15 19:03:10.190703", "rc": 0, "start": "2021-01-15 19:03:08.045297", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:03:10.250 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:10 +0000 (0:00:01.217) 0:03:20.634 ******** 2021-01-15T19:03:10.745 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-01-15T19:03:10.747 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2021-01-15T19:03:11.142 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-15T19:03:11.144 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2021-01-15T19:03:11.181 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:11 +0000 (0:00:00.930) 0:03:21.565 ******** 2021-01-15T19:03:11.197 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-01-15T19:03:11.199 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2021-01-15T19:03:11.637 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2021-01-15T19:03:11.675 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:11 +0000 (0:00:00.494) 0:03:22.060 ******** 2021-01-15T19:03:12.160 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-01-15T19:03:12.162 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2021-01-15T19:03:12.198 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:12 +0000 (0:00:00.522) 0:03:22.583 ******** 2021-01-15T19:03:12.681 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-01-15T19:03:12.682 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2021-01-15T19:03:13.130 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2021-01-15T19:03:13.578 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2021-01-15T19:03:13.622 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:13 +0000 (0:00:01.424) 0:03:24.007 ******** 2021-01-15T19:03:13.652 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-01-15T19:03:13.686 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:13 +0000 (0:00:00.063) 0:03:24.071 ******** 2021-01-15T19:03:13.712 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-01-15T19:03:13.714 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:13.745 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:13 +0000 (0:00:00.059) 0:03:24.130 ******** 2021-01-15T19:03:13.782 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-01-15T19:03:13.784 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-01-15T19:03:13.817 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:13 +0000 (0:00:00.071) 0:03:24.202 ******** 2021-01-15T19:03:13.843 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-01-15T19:03:13.845 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:13.876 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:13 +0000 (0:00:00.059) 0:03:24.261 ******** 2021-01-15T19:03:13.913 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:03:13.915 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2021-01-15T19:03:13.948 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:13 +0000 (0:00:00.071) 0:03:24.333 ******** 2021-01-15T19:03:13.985 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-01-15T19:03:13.986 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:14.018 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:14 +0000 (0:00:00.069) 0:03:24.403 ******** 2021-01-15T19:03:14.047 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-01-15T19:03:14.049 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:14.080 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:14 +0000 (0:00:00.062) 0:03:24.465 ******** 2021-01-15T19:03:14.299 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-01-15T19:03:14.301 INFO:teuthology.task.ansible.out:fatal: [smithi001.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007325", "end": "2021-01-15 19:03:14.273085", "msg": "non-zero return code", "rc": 1, "start": "2021-01-15 19:03:14.265760", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2021-01-15T19:03:14.335 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:14 +0000 (0:00:00.254) 0:03:24.720 ******** 2021-01-15T19:03:14.373 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-01-15T19:03:14.374 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:14.407 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:14 +0000 (0:00:00.072) 0:03:24.792 ******** 2021-01-15T19:03:16.968 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-15T19:03:16.970 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: librbd1-1:12.2.7-9.el8.x86_64", "Removed: qemu-kvm-block-rbd-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64"]} 2021-01-15T19:03:17.001 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:16 +0000 (0:00:02.593) 0:03:27.386 ******** 2021-01-15T19:03:19.188 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-15T19:03:19.190 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-6.el8.x86_64", "Removed: boost-random-1.66.0-6.el8.x86_64"]} 2021-01-15T19:03:19.221 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:19 +0000 (0:00:02.220) 0:03:29.606 ******** 2021-01-15T19:03:52.714 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-15T19:03:52.716 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: util-linux-2.32.1-24.el8.x86_64", "Installed: util-linux-user-2.32.1-24.el8.x86_64", "Installed: python3-libdnf-0.48.0-5.el8.x86_64", "Installed: libdnf-0.48.0-5.el8.x86_64", "Installed: python3-librepo-1.12.0-2.el8.x86_64", "Installed: boost-chrono-1.66.0-10.el8.x86_64", "Installed: xfsprogs-5.0.0-4.el8.x86_64", "Installed: libfdisk-2.32.1-24.el8.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: gdb-8.2-12.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: nfs-utils-1:2.3.3-35.el8.x86_64", "Installed: gcc-8.3.1-5.1.el8.x86_64", "Installed: gdb-headless-8.2-12.el8.x86_64", "Installed: libuuid-2.32.1-24.el8.x86_64", "Installed: boost-program-options-1.66.0-10.el8.x86_64", "Installed: gcc-gdb-plugin-8.3.1-5.1.el8.x86_64", "Installed: boost-random-1.66.0-10.el8.x86_64", "Installed: python3-virtualenv-15.1.0-19.module_el8.3.0+562+e162826a.noarch", "Installed: libgcc-8.3.1-5.1.el8.x86_64", "Installed: libgcrypt-1.8.5-4.el8.x86_64", "Installed: yum-utils-4.0.17-5.el8.no 2021-01-15T19:03:52.718 INFO:teuthology.task.ansible.out:arch", "Installed: libgomp-8.3.1-5.1.el8.x86_64", "Installed: yum-4.2.23-4.el8.noarch", "Installed: perl-Git-2.27.0-1.el8.noarch", "Installed: boost-system-1.66.0-10.el8.x86_64", "Installed: boost-thread-1.66.0-10.el8.x86_64", "Installed: librepo-1.12.0-2.el8.x86_64", "Installed: podman-2.0.5-5.module_el8.3.0+512+b3b58dca.x86_64", "Installed: podman-catatonit-2.0.5-5.module_el8.3.0+512+b3b58dca.x86_64", "Installed: podman-docker-2.0.5-5.module_el8.3.0+512+b3b58dca.noarch", "Installed: dnf-4.2.23-4.el8.noarch", "Installed: dnf-data-4.2.23-4.el8.noarch", "Installed: dnf-plugins-core-4.0.17-5.el8.noarch", "Installed: python3-nose-1.3.7-30.module_el8.3.0+562+e162826a.noarch", "Installed: libsmartcols-2.32.1-24.el8.x86_64", "Installed: openssl-1:1.1.1g-12.el8_3.x86_64", "Installed: lsof-4.93.2-1.el8.x86_64", "Installed: valgrind-1:3.16.0-2.el8.x86_64", "Installed: libblkid-2.32.1-24.el8.x86_64", "Installed: git-2.27.0-1.el8.x86_64", "Installed: openssl-libs-1:1.1.1g-12.el8_3.x86_64", "Installed: git-core-2.27.0-1.el8.x86_64", "Installed: git-core-doc-2.27.0-1.el8.noarch", "Installed: qemu-img-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: qemu-kvm-block-rbd-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: qemu-kvm-common-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: qemu-kvm-core-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: python3-hawkey-0.48.0-5.el8.x86_64", "Installed: cpp-8.3.1-5.1.el8.x86_64", "Installed: libmodulemd-2.9.4-2.el8.x86_64", "Installed: libmount-2.32.1-24.el8.x86_64", "Installed: python3-dnf-4.2.23-4.el8.noarch", "Installed: python3-dnf-plugins-core-4.0.17-5.el8.noarch", "Installed: sysstat-11.7.3-5.el8.x86_64", "Installed: seavgabios-bin-1.13.0-2.module_el8.3.0+555+a55c8938.noarch", "Removed: python3-virtualenv-15.1.0-19.module_el8.1.0+245+c39af44f.noarch", "Removed: openssl-1:1.1.1c-15.el8.x86_64", "Removed: qemu-img-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: qemu-kvm- 2021-01-15T19:03:52.719 INFO:teuthology.task.ansible.out:common-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: qemu-kvm-core-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: openssl-libs-1:1.1.1c-15.el8.x86_64", "Removed: librepo-1.11.0-2.el8.x86_64", "Removed: libsmartcols-2.32.1-22.el8.x86_64", "Removed: gcc-8.3.1-4.5.el8.x86_64", "Removed: gcc-gdb-plugin-8.3.1-4.5.el8.x86_64", "Removed: gdb-8.2-6.el8.x86_64", "Removed: gdb-headless-8.2-6.el8.x86_64", "Removed: seavgabios-bin-1.11.1-4.module_el8.1.0+248+298dec18.noarch", "Removed: boost-chrono-1.66.0-6.el8.x86_64", "Removed: git-2.18.2-2.el8_1.x86_64", "Removed: git-core-2.18.2-2.el8_1.x86_64", "Removed: git-core-doc-2.18.2-2.el8_1.noarch", "Removed: boost-system-1.66.0-6.el8.x86_64", "Removed: boost-thread-1.66.0-6.el8.x86_64", "Removed: libuuid-2.32.1-22.el8.x86_64", "Removed: podman-1.6.4-4.module_el8.1.0+298+41f9343a.x86_64", "Removed: podman-docker-1.6.4-4.module_el8.1.0+298+41f9343a.noarch", "Removed: podman-manpages-1.6.4-4.module_el8.1.0+298+41f9343a.noarch", "Removed: libblkid-2.32.1-22.el8.x86_64", "Removed: libdnf-0.35.1-9.el8_1.x86_64", "Removed: perl-Git-2.18.2-2.el8_1.noarch", "Removed: lsof-4.91-2.el8.x86_64", "Removed: cpp-8.3.1-4.5.el8.x86_64", "Removed: sysstat-11.7.3-2.el8.x86_64", "Removed: libfdisk-2.32.1-22.el8.x86_64", "Removed: python3-dnf-4.2.7-7.el8_1.noarch", "Removed: python3-dnf-plugins-core-4.0.8-3.el8.noarch", "Removed: libgcc-8.3.1-5.el8.0.2.x86_64", "Removed: libgcrypt-1.8.3-4.el8.x86_64", "Removed: python3-hawkey-0.35.1-9.el8_1.x86_64", "Removed: libgomp-8.3.1-4.5.el8.x86_64", "Removed: python3-libdnf-0.35.1-9.el8_1.x86_64", "Removed: python3-librepo-1.11.0-2.el8.x86_64", "Removed: util-linux-2.32.1-22.el8.x86_64", "Removed: util-linux-user-2.32.1-22.el8.x86_64", "Removed: valgrind-1:3.15.0-9.el8.x86_64", "Removed: python3-nose-1.3.7-30.module_el8.1.0+245+c39af44f.noarch", "Removed: dnf-4.2.7-7.el8_1.noarch", "Removed: dnf-data-4.2.7-7.el8_1.noarch", "Removed: libmount-2.32.1-22.el8.x86_64", "Removed: dnf-plu 2021-01-15T19:03:52.721 INFO:teuthology.task.ansible.out:gins-core-4.0.8-3.el8.noarch", "Removed: nfs-utils-1:2.3.3-26.el8.x86_64", "Removed: xfsprogs-5.0.0-2.el8.x86_64", "Removed: yum-4.2.7-7.el8_1.noarch", "Removed: yum-utils-4.0.8-3.el8.noarch"]} 2021-01-15T19:03:52.755 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:52 +0000 (0:00:33.533) 0:04:03.140 ******** 2021-01-15T19:03:52.783 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-01-15T19:03:52.785 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:52.819 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:52 +0000 (0:00:00.063) 0:04:03.204 ******** 2021-01-15T19:03:53.943 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-01-15T19:03:53.945 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-01-15T19:03:53.976 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:53 +0000 (0:00:01.156) 0:04:04.361 ******** 2021-01-15T19:03:54.002 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-15T19:03:54.004 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:54.035 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:54 +0000 (0:00:00.059) 0:04:04.420 ******** 2021-01-15T19:03:54.342 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-01-15T19:03:54.343 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "dbus.service sysinit.target system.slice polkit.service basic.target dbus.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ipset.service ip6tables.service ebtables.service shutdown.target iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNode 2021-01-15T19:03:54.345 INFO:teuthology.task.ansible.out:s": "", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127261", "LimitNPROCSoft": "127261", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "L 2021-01-15T19:03:54.346 INFO:teuthology.task.ansible.out:imitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127261", "LimitSIGPENDINGSoft": "127261", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket system.slice sysinit.target", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWe 2021-01-15T19:03:54.348 INFO:teuthology.task.ansible.out:ight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "203618", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-01-15T19:03:54.377 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:54 +0000 (0:00:00.342) 0:04:04.762 ******** 2021-01-15T19:03:54.664 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-01-15T19:03:54.666 INFO:teuthology.task.ansible.out:fatal: [smithi136.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2021-01-15T19:03:54.699 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:54 +0000 (0:00:00.321) 0:04:05.084 ******** 2021-01-15T19:03:55.393 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2021-01-15T19:03:55.427 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.727) 0:04:05.812 ******** 2021-01-15T19:03:55.454 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-01-15T19:03:55.456 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.489 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.062) 0:04:05.874 ******** 2021-01-15T19:03:55.516 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-01-15T19:03:55.518 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.551 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.061) 0:04:05.936 ******** 2021-01-15T19:03:55.577 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-01-15T19:03:55.579 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.614 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.063) 0:04:05.999 ******** 2021-01-15T19:03:55.641 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-01-15T19:03:55.642 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.677 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.062) 0:04:06.062 ******** 2021-01-15T19:03:55.705 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-01-15T19:03:55.707 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.739 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.061) 0:04:06.123 ******** 2021-01-15T19:03:55.766 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.797 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.058) 0:04:06.182 ******** 2021-01-15T19:03:55.823 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-01-15T19:03:55.855 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.057) 0:04:06.240 ******** 2021-01-15T19:03:55.883 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-01-15T19:03:55.884 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.931 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.075) 0:04:06.316 ******** 2021-01-15T19:03:55.965 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:55.997 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:55 +0000 (0:00:00.066) 0:04:06.382 ******** 2021-01-15T19:03:56.046 INFO:teuthology.task.ansible.out: 2021-01-15T19:03:56.048 INFO:teuthology.task.ansible.out:TASK [testnode : Install apt keys] ********************************************* skipping: [smithi136.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.058 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.093 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.096) 0:04:06.478 ******** 2021-01-15T19:03:56.119 INFO:teuthology.task.ansible.out: 2021-01-15T19:03:56.121 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-apt] ******************************************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.157 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.063) 0:04:06.541 ******** 2021-01-15T19:03:56.183 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-01-15T19:03:56.218 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.061) 0:04:06.603 ******** 2021-01-15T19:03:56.250 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.280 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.061) 0:04:06.665 ******** 2021-01-15T19:03:56.318 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.355 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.074) 0:04:06.739 ******** 2021-01-15T19:03:56.391 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-01-15T19:03:56.393 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.426 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.071) 0:04:06.810 ******** 2021-01-15T19:03:56.454 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.488 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.062) 0:04:06.873 ******** 2021-01-15T19:03:56.514 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-01-15T19:03:56.516 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.548 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.060) 0:04:06.933 ******** 2021-01-15T19:03:56.585 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.619 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.070) 0:04:07.004 ******** 2021-01-15T19:03:56.658 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-01-15T19:03:56.660 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.691 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.072) 0:04:07.076 ******** 2021-01-15T19:03:56.725 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-01-15T19:03:56.727 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.769 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.077) 0:04:07.154 ******** 2021-01-15T19:03:56.796 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.829 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.060) 0:04:07.214 ******** 2021-01-15T19:03:56.855 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-01-15T19:03:56.857 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.894 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.064) 0:04:07.279 ******** 2021-01-15T19:03:56.920 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-01-15T19:03:56.922 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:56.954 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:56 +0000 (0:00:00.059) 0:04:07.338 ******** 2021-01-15T19:03:56.980 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-01-15T19:03:56.982 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.013 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:07.398 ******** 2021-01-15T19:03:57.040 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.073 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:07.458 ******** 2021-01-15T19:03:57.114 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-01-15T19:03:57.116 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.147 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.073) 0:04:07.531 ******** 2021-01-15T19:03:57.173 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-01-15T19:03:57.175 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.206 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:07.591 ******** 2021-01-15T19:03:57.233 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-15T19:03:57.235 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.267 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.060) 0:04:07.652 ******** 2021-01-15T19:03:57.292 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-15T19:03:57.294 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.328 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.060) 0:04:07.712 ******** 2021-01-15T19:03:57.360 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-01-15T19:03:57.362 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.395 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.067) 0:04:07.780 ******** 2021-01-15T19:03:57.423 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-01-15T19:03:57.425 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.460 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.064) 0:04:07.844 ******** 2021-01-15T19:03:57.488 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-01-15T19:03:57.490 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.522 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.062) 0:04:07.907 ******** 2021-01-15T19:03:57.548 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-15T19:03:57.550 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.581 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:07.966 ******** 2021-01-15T19:03:57.608 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-15T19:03:57.609 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.641 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:08.026 ******** 2021-01-15T19:03:57.667 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:03:57.669 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.701 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:08.085 ******** 2021-01-15T19:03:57.726 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-01-15T19:03:57.727 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.759 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.058) 0:04:08.144 ******** 2021-01-15T19:03:57.785 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-15T19:03:57.786 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.818 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.059) 0:04:08.203 ******** 2021-01-15T19:03:57.847 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-15T19:03:57.849 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.880 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.061) 0:04:08.265 ******** 2021-01-15T19:03:57.917 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-15T19:03:57.919 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:57.951 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:57 +0000 (0:00:00.071) 0:04:08.336 ******** 2021-01-15T19:03:58.169 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-15T19:03:58.171 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2021-01-15T19:03:58.203 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.252) 0:04:08.588 ******** 2021-01-15T19:03:58.417 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-15T19:03:58.419 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:03:58.454 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.250) 0:04:08.839 ******** 2021-01-15T19:03:58.649 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2021-01-15T19:03:58.680 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.226) 0:04:09.065 ******** 2021-01-15T19:03:58.707 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:58.742 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.061) 0:04:09.127 ******** 2021-01-15T19:03:58.769 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-15T19:03:58.771 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:58.804 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.062) 0:04:09.189 ******** 2021-01-15T19:03:58.831 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-15T19:03:58.833 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:58.870 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.065) 0:04:09.255 ******** 2021-01-15T19:03:58.898 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-15T19:03:58.900 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:58.934 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:58 +0000 (0:00:00.064) 0:04:09.319 ******** 2021-01-15T19:03:58.982 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-15T19:03:58.984 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.018 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.083) 0:04:09.402 ******** 2021-01-15T19:03:59.047 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-15T19:03:59.049 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.082 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.064) 0:04:09.467 ******** 2021-01-15T19:03:59.109 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-15T19:03:59.111 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.142 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.060) 0:04:09.527 ******** 2021-01-15T19:03:59.169 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-15T19:03:59.171 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.202 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.059) 0:04:09.587 ******** 2021-01-15T19:03:59.229 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-15T19:03:59.230 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.262 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.059) 0:04:09.647 ******** 2021-01-15T19:03:59.288 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-01-15T19:03:59.290 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.325 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.063) 0:04:09.710 ******** 2021-01-15T19:03:59.351 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-01-15T19:03:59.352 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.385 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.060) 0:04:09.770 ******** 2021-01-15T19:03:59.413 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.447 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.061) 0:04:09.832 ******** 2021-01-15T19:03:59.473 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.507 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.059) 0:04:09.892 ******** 2021-01-15T19:03:59.534 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-01-15T19:03:59.536 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.568 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.060) 0:04:09.953 ******** 2021-01-15T19:03:59.594 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-01-15T19:03:59.596 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.630 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.061) 0:04:10.014 ******** 2021-01-15T19:03:59.657 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-01-15T19:03:59.660 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.692 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.062) 0:04:10.077 ******** 2021-01-15T19:03:59.719 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-01-15T19:03:59.721 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.754 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.061) 0:04:10.139 ******** 2021-01-15T19:03:59.781 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-01-15T19:03:59.782 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.817 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.062) 0:04:10.202 ******** 2021-01-15T19:03:59.844 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-01-15T19:03:59.846 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.881 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.063) 0:04:10.266 ******** 2021-01-15T19:03:59.908 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:03:59.946 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:03:59 +0000 (0:00:00.065) 0:04:10.331 ******** 2021-01-15T19:03:59.973 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-01-15T19:03:59.974 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.010 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.063) 0:04:10.395 ******** 2021-01-15T19:04:00.036 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-01-15T19:04:00.038 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.070 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.060) 0:04:10.455 ******** 2021-01-15T19:04:00.098 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-01-15T19:04:00.131 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.060) 0:04:10.516 ******** 2021-01-15T19:04:00.178 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-01-15T19:04:00.179 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.190 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-01-15T19:04:00.192 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.201 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.212 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.213 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.224 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.266 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.134) 0:04:10.650 ******** 2021-01-15T19:04:00.267 INFO:teuthology.task.ansible.out: 2021-01-15T19:04:00.450 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi136.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003473", "end": "2021-01-15 19:04:00.425175", "rc": 0, "start": "2021-01-15 19:04:00.421702", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2021-01-15T19:04:00.489 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.223) 0:04:10.874 ******** 2021-01-15T19:04:00.531 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-01-15T19:04:00.533 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.568 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.079) 0:04:10.953 ******** 2021-01-15T19:04:00.595 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-01-15T19:04:00.596 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.633 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.065) 0:04:11.018 ******** 2021-01-15T19:04:00.664 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-01-15T19:04:00.666 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.699 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.065) 0:04:11.084 ******** 2021-01-15T19:04:00.726 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-01-15T19:04:00.728 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:00.763 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:00 +0000 (0:00:00.064) 0:04:11.148 ******** 2021-01-15T19:04:03.166 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-15T19:04:03.168 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: podman", "Installed: podman-docker", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: dbench", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: util-linux-2.32.1-24.el8.x86_64", "Installed: util-linux-user-2.32.1-24.el8.x86_64", "Installed: python3-libdnf-0.48.0-5.el8.x86_64", "Installed: libdnf-0.48.0-5.el8.x86_64", "Installed: python3-librepo-1.12.0-2.el8.x86_64", "Installed: boost-chrono-1.66.0-10.el8.x86_64", "Installed: xfsprogs-5.0.0-4.el8.x86_64", "Installed: libfdisk-2.32.1-24.el8.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: gdb-8.2-12.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: nfs-utils-1:2.3.3-35.el8.x86_64", "Installed: gcc-8.3.1-5.1.el8.x86_64", "Installed: gdb-headless-8.2-12.el8.x86_64", "Installed: libuuid-2.32.1-24.el8.x86_64", "Installed: boost-program-options-1.66.0-10.el8.x86_64", "Installed: gcc-gdb-plugin-8.3.1-5.1.el8.x86_64", "Installed: boost-random-1.66.0-10.el8.x86_64", "Installed: python3-virtualenv-15.1.0-19.module_el8.3.0+562+e162826a.noarch", "Installed: libgcc-8.3.1-5.1.el8.x86_64", "Installed: libgcrypt-1.8.5-4.el8.x86_64", "Installed: yum-utils-4.0.17-5.el8.no 2021-01-15T19:04:03.170 INFO:teuthology.task.ansible.out:arch", "Installed: libgomp-8.3.1-5.1.el8.x86_64", "Installed: yum-4.2.23-4.el8.noarch", "Installed: perl-Git-2.27.0-1.el8.noarch", "Installed: boost-system-1.66.0-10.el8.x86_64", "Installed: boost-thread-1.66.0-10.el8.x86_64", "Installed: librepo-1.12.0-2.el8.x86_64", "Installed: podman-2.0.5-5.module_el8.3.0+512+b3b58dca.x86_64", "Installed: podman-catatonit-2.0.5-5.module_el8.3.0+512+b3b58dca.x86_64", "Installed: podman-docker-2.0.5-5.module_el8.3.0+512+b3b58dca.noarch", "Installed: dnf-4.2.23-4.el8.noarch", "Installed: dnf-data-4.2.23-4.el8.noarch", "Installed: dnf-plugins-core-4.0.17-5.el8.noarch", "Installed: python3-nose-1.3.7-30.module_el8.3.0+562+e162826a.noarch", "Installed: libsmartcols-2.32.1-24.el8.x86_64", "Installed: openssl-1:1.1.1g-12.el8_3.x86_64", "Installed: lsof-4.93.2-1.el8.x86_64", "Installed: valgrind-1:3.16.0-2.el8.x86_64", "Installed: libblkid-2.32.1-24.el8.x86_64", "Installed: git-2.27.0-1.el8.x86_64", "Installed: openssl-libs-1:1.1.1g-12.el8_3.x86_64", "Installed: git-core-2.27.0-1.el8.x86_64", "Installed: git-core-doc-2.27.0-1.el8.noarch", "Installed: qemu-img-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: qemu-kvm-block-rbd-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: qemu-kvm-common-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: qemu-kvm-core-15:4.2.0-34.module_el8.3.0+613+9ec9f184.1.x86_64", "Installed: python3-hawkey-0.48.0-5.el8.x86_64", "Installed: cpp-8.3.1-5.1.el8.x86_64", "Installed: libmodulemd-2.9.4-2.el8.x86_64", "Installed: libmount-2.32.1-24.el8.x86_64", "Installed: python3-dnf-4.2.23-4.el8.noarch", "Installed: python3-dnf-plugins-core-4.0.17-5.el8.noarch", "Installed: sysstat-11.7.3-5.el8.x86_64", "Installed: seavgabios-bin-1.13.0-2.module_el8.3.0+555+a55c8938.noarch", "Removed: python3-virtualenv-15.1.0-19.module_el8.1.0+245+c39af44f.noarch", "Removed: openssl-1:1.1.1c-15.el8.x86_64", "Removed: qemu-img-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: qemu-kvm- 2021-01-15T19:04:03.171 INFO:teuthology.task.ansible.out:common-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: qemu-kvm-core-15:2.12.0-88.module_el8.1.0+297+df420408.3.x86_64", "Removed: openssl-libs-1:1.1.1c-15.el8.x86_64", "Removed: librepo-1.11.0-2.el8.x86_64", "Removed: libsmartcols-2.32.1-22.el8.x86_64", "Removed: gcc-8.3.1-4.5.el8.x86_64", "Removed: gcc-gdb-plugin-8.3.1-4.5.el8.x86_64", "Removed: gdb-8.2-6.el8.x86_64", "Removed: gdb-headless-8.2-6.el8.x86_64", "Removed: seavgabios-bin-1.11.1-4.module_el8.1.0+248+298dec18.noarch", "Removed: boost-chrono-1.66.0-6.el8.x86_64", "Removed: git-2.18.2-2.el8_1.x86_64", "Removed: git-core-2.18.2-2.el8_1.x86_64", "Removed: git-core-doc-2.18.2-2.el8_1.noarch", "Removed: boost-system-1.66.0-6.el8.x86_64", "Removed: boost-thread-1.66.0-6.el8.x86_64", "Removed: libuuid-2.32.1-22.el8.x86_64", "Removed: podman-1.6.4-4.module_el8.1.0+298+41f9343a.x86_64", "Removed: podman-docker-1.6.4-4.module_el8.1.0+298+41f9343a.noarch", "Removed: podman-manpages-1.6.4-4.module_el8.1.0+298+41f9343a.noarch", "Removed: libblkid-2.32.1-22.el8.x86_64", "Removed: libdnf-0.35.1-9.el8_1.x86_64", "Removed: perl-Git-2.18.2-2.el8_1.noarch", "Removed: lsof-4.91-2.el8.x86_64", "Removed: cpp-8.3.1-4.5.el8.x86_64", "Removed: sysstat-11.7.3-2.el8.x86_64", "Removed: libfdisk-2.32.1-22.el8.x86_64", "Removed: python3-dnf-4.2.7-7.el8_1.noarch", "Removed: python3-dnf-plugins-core-4.0.8-3.el8.noarch", "Removed: libgcc-8.3.1-5.el8.0.2.x86_64", "Removed: libgcrypt-1.8.3-4.el8.x86_64", "Removed: python3-hawkey-0.35.1-9.el8_1.x86_64", "Removed: libgomp-8.3.1-4.5.el8.x86_64", "Removed: python3-libdnf-0.35.1-9.el8_1.x86_64", "Removed: python3-librepo-1.11.0-2.el8.x86_64", "Removed: util-linux-2.32.1-22.el8.x86_64", "Removed: util-linux-user-2.32.1-22.el8.x86_64", "Removed: valgrind-1:3.15.0-9.el8.x86_64", "Removed: python3-nose-1.3.7-30.module_el8.1.0+245+c39af44f.noarch", "Removed: dnf-4.2.7-7.el8_1.noarch", "Removed: dnf-data-4.2.7-7.el8_1.noarch", "Removed: libmount-2.32.1-22.el8.x86_64", "Removed: dnf-plu 2021-01-15T19:04:03.173 INFO:teuthology.task.ansible.out:gins-core-4.0.8-3.el8.noarch", "Removed: nfs-utils-1:2.3.3-26.el8.x86_64", "Removed: xfsprogs-5.0.0-2.el8.x86_64", "Removed: yum-4.2.7-7.el8_1.noarch", "Removed: yum-utils-4.0.8-3.el8.noarch"]} 2021-01-15T19:04:03.207 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:03 +0000 (0:00:02.443) 0:04:13.591 ******** 2021-01-15T19:04:03.235 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-01-15T19:04:03.237 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:03.279 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:03 +0000 (0:00:00.072) 0:04:13.664 ******** 2021-01-15T19:04:04.458 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-01-15T19:04:04.460 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-01-15T19:04:04.498 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:04 +0000 (0:00:01.218) 0:04:14.882 ******** 2021-01-15T19:04:04.525 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-15T19:04:04.527 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:04.566 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:04 +0000 (0:00:00.068) 0:04:14.951 ******** 2021-01-15T19:04:04.890 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-01-15T19:04:04.891 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target system.slice sysinit.target dbus.socket dbus.service polkit.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target ipset.service ip6tables.service ebtables.service iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNode 2021-01-15T19:04:04.892 INFO:teuthology.task.ansible.out:s": "", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127286", "LimitNPROCSoft": "127286", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "L 2021-01-15T19:04:04.894 INFO:teuthology.task.ansible.out:imitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127286", "LimitSIGPENDINGSoft": "127286", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice dbus.socket", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWe 2021-01-15T19:04:04.896 INFO:teuthology.task.ansible.out:ight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "203657", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-01-15T19:04:04.928 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:04 +0000 (0:00:00.361) 0:04:15.312 ******** 2021-01-15T19:04:05.233 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-01-15T19:04:05.235 INFO:teuthology.task.ansible.out:fatal: [smithi001.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2021-01-15T19:04:05.268 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.340) 0:04:15.653 ******** 2021-01-15T19:04:05.559 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-01-15T19:04:05.561 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2021-01-15T19:04:05.597 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.328) 0:04:15.982 ******** 2021-01-15T19:04:05.625 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-01-15T19:04:05.627 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:05.659 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.062) 0:04:16.044 ******** 2021-01-15T19:04:05.702 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-01-15T19:04:05.704 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:05.738 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.078) 0:04:16.123 ******** 2021-01-15T19:04:05.765 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-01-15T19:04:05.767 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure rpm dependencies are installed] ********************* changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper", "Installed: device-mapper-event-8:1.02.171-5.el8.x86_64", "Installed: lvm2-8:2.03.09-5.el8.x86_64", "Installed: device-mapper-event-libs-8:1.02.171-5.el8.x86_64", "Installed: lvm2-libs-8:2.03.09-5.el8.x86_64", "Installed: device-mapper-libs-8:1.02.171-5.el8.x86_64", "Installed: device-mapper-8:1.02.171-5.el8.x86_64", "Removed: device-mapper-8:1.02.169-3.el8.x86_64", "Removed: device-mapper-event-8:1.02.169-3.el8.x86_64", "Removed: device-mapper-event-libs-8:1.02.169-3.el8.x86_64", "Removed: device-mapper-libs-8:1.02.169-3.el8.x86_64", "Removed: lvm2-8:2.03.08-3.el8.x86_64", "Removed: lvm2-libs-8:2.03.08-3.el8.x86_64"]} 2021-01-15T19:04:05.802 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.064) 0:04:16.187 ******** 2021-01-15T19:04:05.837 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.034) 0:04:16.222 ******** 2021-01-15T19:04:05.844 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-01-15T19:04:05.845 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:05.882 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.045) 0:04:16.267 ******** 2021-01-15T19:04:05.889 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-15T19:04:05.891 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930356453376, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227137806, 'block_used': 12957302, 'inode_total': 61054976, 'inode_available': 60986349, 'inode_used': 68627, 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227137806, "block_size": 4096, "block_total": 240095108, "block_used": 12957302, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986349, "inode_total": 61054976, "inode_used": 68627, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930356453376, "size_total": 983429562368, "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}} 2021-01-15T19:04:05.929 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.046) 0:04:16.314 ******** 2021-01-15T19:04:05.936 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-01-15T19:04:05.938 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:05.977 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:05 +0000 (0:00:00.048) 0:04:16.362 ******** 2021-01-15T19:04:06.006 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-01-15T19:04:06.008 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.041 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.064) 0:04:16.426 ******** 2021-01-15T19:04:06.068 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-01-15T19:04:06.124 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.082) 0:04:16.509 ******** 2021-01-15T19:04:06.134 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-01-15T19:04:06.135 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005187", "end": "2021-01-15 19:04:06.104787", "rc": 0, "start": "2021-01-15 19:04:06.099600", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:06.190 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.066) 0:04:16.575 ******** 2021-01-15T19:04:06.199 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-01-15T19:04:06.201 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.240 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.049) 0:04:16.624 ******** 2021-01-15T19:04:06.248 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-01-15T19:04:06.248 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930356453376, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227137806, 'block_used': 12957302, 'inode_total': 61054976, 'inode_available': 60986349, 'inode_used': 68627, 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227137806, "block_size": 4096, "block_total": 240095108, "block_used": 12957302, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986349, "inode_total": 61054976, "inode_used": 68627, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930356453376, "size_total": 983429562368, "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.287 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.047) 0:04:16.672 ******** 2021-01-15T19:04:06.295 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-01-15T19:04:06.297 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.330 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.043) 0:04:16.715 ******** 2021-01-15T19:04:06.339 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-01-15T19:04:06.379 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.048) 0:04:16.764 ******** 2021-01-15T19:04:06.385 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-01-15T19:04:06.387 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.426 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.046) 0:04:16.811 ******** 2021-01-15T19:04:06.453 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-01-15T19:04:06.455 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.488 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.062) 0:04:16.873 ******** 2021-01-15T19:04:06.513 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-01-15T19:04:06.545 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.057) 0:04:16.930 ******** 2021-01-15T19:04:06.573 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.607 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.061) 0:04:16.992 ******** 2021-01-15T19:04:06.635 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-01-15T19:04:06.637 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.672 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.064) 0:04:17.057 ******** 2021-01-15T19:04:06.700 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-01-15T19:04:06.702 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.750 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.077) 0:04:17.134 ******** 2021-01-15T19:04:06.778 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-15T19:04:06.780 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.812 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.062) 0:04:17.197 ******** 2021-01-15T19:04:06.848 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-01-15T19:04:06.850 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.883 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.071) 0:04:17.268 ******** 2021-01-15T19:04:06.912 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-15T19:04:06.914 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:06.944 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:06 +0000 (0:00:00.061) 0:04:17.329 ******** 2021-01-15T19:04:06.971 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-01-15T19:04:06.973 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.005 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.060) 0:04:17.390 ******** 2021-01-15T19:04:07.031 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-01-15T19:04:07.033 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.065 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.060) 0:04:17.450 ******** 2021-01-15T19:04:07.093 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.132 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.066) 0:04:17.517 ******** 2021-01-15T19:04:07.159 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-01-15T19:04:07.161 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.202 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.070) 0:04:17.587 ******** 2021-01-15T19:04:07.229 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-01-15T19:04:07.231 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.280 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.077) 0:04:17.664 ******** 2021-01-15T19:04:07.311 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-01-15T19:04:07.313 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.366 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.085) 0:04:17.750 ******** 2021-01-15T19:04:07.397 INFO:teuthology.task.ansible.out: 2021-01-15T19:04:07.398 INFO:teuthology.task.ansible.out:TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.436 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.070) 0:04:17.821 ******** 2021-01-15T19:04:07.475 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-01-15T19:04:07.477 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.530 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.093) 0:04:17.914 ******** 2021-01-15T19:04:07.557 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-01-15T19:04:07.559 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.597 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.066) 0:04:17.981 ******** 2021-01-15T19:04:07.622 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-01-15T19:04:07.624 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.659 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.063) 0:04:18.044 ******** 2021-01-15T19:04:07.689 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-01-15T19:04:07.690 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.726 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.066) 0:04:18.111 ******** 2021-01-15T19:04:07.753 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-01-15T19:04:07.755 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.786 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.059) 0:04:18.171 ******** 2021-01-15T19:04:07.813 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-01-15T19:04:07.815 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.851 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.065) 0:04:18.236 ******** 2021-01-15T19:04:07.880 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-01-15T19:04:07.882 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.916 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.064) 0:04:18.301 ******** 2021-01-15T19:04:07.943 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-01-15T19:04:07.945 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:07.984 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:07 +0000 (0:00:00.067) 0:04:18.369 ******** 2021-01-15T19:04:08.011 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-01-15T19:04:08.013 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:08.047 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.062) 0:04:18.431 ******** 2021-01-15T19:04:08.073 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:08.075 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:08.109 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.061) 0:04:18.493 ******** 2021-01-15T19:04:08.135 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:08.171 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.062) 0:04:18.556 ******** 2021-01-15T19:04:08.199 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-01-15T19:04:08.200 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:08.234 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.063) 0:04:18.619 ******** 2021-01-15T19:04:08.261 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-01-15T19:04:08.263 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:08.297 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.062) 0:04:18.682 ******** 2021-01-15T19:04:08.327 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-15T19:04:08.329 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:08.361 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.064) 0:04:18.746 ******** 2021-01-15T19:04:08.400 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:08.402 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CApUEvJfzzHbbAvKVy5ZTEJVXX6XlcgjEj'], 'uuids': ['d8d2c0b2-32e6-4f01-8b2e-5c0dd295b85c'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.795825", "end": "2021-01-15 19:04:08.366304", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CApUEvJfzzHbbAvKVy5ZTEJVXX6XlcgjEj"], "labels": [], "masters": [], "uuids": ["d8d2c0b2-32e6-4f01-8b2e-5c0dd295b85c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:06.570479", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:08.594 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-15T19:04:08.595 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2021-01-15T19:04:08.628 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.266) 0:04:19.013 ******** 2021-01-15T19:04:08.856 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-15T19:04:08.857 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:08.892 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:08 +0000 (0:00:00.263) 0:04:19.277 ******** 2021-01-15T19:04:09.099 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2021-01-15T19:04:09.133 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.241) 0:04:19.518 ******** 2021-01-15T19:04:09.160 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-01-15T19:04:09.161 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.195 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.061) 0:04:19.580 ******** 2021-01-15T19:04:09.222 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-15T19:04:09.224 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.261 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.065) 0:04:19.645 ******** 2021-01-15T19:04:09.287 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-01-15T19:04:09.289 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.322 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.061) 0:04:19.707 ******** 2021-01-15T19:04:09.350 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-15T19:04:09.351 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.383 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.061) 0:04:19.768 ******** 2021-01-15T19:04:09.410 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-15T19:04:09.412 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.445 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.062) 0:04:19.830 ******** 2021-01-15T19:04:09.484 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-15T19:04:09.485 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.517 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.071) 0:04:19.902 ******** 2021-01-15T19:04:09.543 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-01-15T19:04:09.545 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.577 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.060) 0:04:19.962 ******** 2021-01-15T19:04:09.604 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-01-15T19:04:09.606 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.639 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.062) 0:04:20.024 ******** 2021-01-15T19:04:09.666 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-01-15T19:04:09.668 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.701 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.061) 0:04:20.086 ******** 2021-01-15T19:04:09.708 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:09.709 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-aAPi1K-INVF-Xq05-weoS-FinR-cP0G-6rfBsr', 'nvme-INTEL_SSDPEDMX020T4_CVPD514100122P0SGN', 'nvme-nvme.8086-43565044353134313030313232503053474e-494e54454c205353445045444d583032305434-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMX020T4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '3907029168', 'sectorsize': '512', 'size': '1.82 TB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.091884", "end": "2021-01-15 19:04:09.648981", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-aAPi1K-INVF-Xq05-weoS-FinR-cP0G-6rfBsr", "nvme-INTEL_SSDPEDMX020T4_CVPD514100122P0SGN", "nvme-nvme.8086-43565044353134313030313232503053474e-494e54454c205353445045444d583032305434-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMX020T4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "3907029168", "sectorsize": "512", "size": "1.82 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:08.557097", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! Yo 2021-01-15T19:04:09.711 INFO:teuthology.task.ansible.out:u may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:09.720 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.756 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.055) 0:04:20.141 ******** 2021-01-15T19:04:09.793 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-01-15T19:04:09.795 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.853 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.096) 0:04:20.238 ******** 2021-01-15T19:04:09.884 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-01-15T19:04:09.886 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.920 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.067) 0:04:20.305 ******** 2021-01-15T19:04:09.947 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-01-15T19:04:09.949 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:09.983 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:09 +0000 (0:00:00.061) 0:04:20.367 ******** 2021-01-15T19:04:10.009 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-01-15T19:04:10.011 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.043 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.060) 0:04:20.428 ******** 2021-01-15T19:04:10.070 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-01-15T19:04:10.072 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.104 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.061) 0:04:20.489 ******** 2021-01-15T19:04:10.133 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-01-15T19:04:10.134 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.166 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.061) 0:04:20.551 ******** 2021-01-15T19:04:10.193 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-01-15T19:04:10.195 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.227 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.060) 0:04:20.612 ******** 2021-01-15T19:04:10.253 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-01-15T19:04:10.255 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.286 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.058) 0:04:20.670 ******** 2021-01-15T19:04:10.312 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-01-15T19:04:10.313 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.344 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.058) 0:04:20.729 ******** 2021-01-15T19:04:10.370 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-01-15T19:04:10.372 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.402 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.057) 0:04:20.787 ******** 2021-01-15T19:04:10.428 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-01-15T19:04:10.430 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.460 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.057) 0:04:20.844 ******** 2021-01-15T19:04:10.487 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-01-15T19:04:10.489 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.518 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.058) 0:04:20.903 ******** 2021-01-15T19:04:10.543 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:10.545 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-01-15T19:04:10.575 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.056) 0:04:20.959 ******** 2021-01-15T19:04:10.624 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-01-15T19:04:10.625 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.635 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-01-15T19:04:10.645 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.647 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.656 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.667 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.669 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:10.713 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.137) 0:04:21.097 ******** 2021-01-15T19:04:10.908 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:10.910 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAc2toRkPyFMTV8MZv8YiRs7dKWOaGqPzO'], 'uuids': ['803396ec-d322-45bb-b215-e297daac7584'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.040913", "end": "2021-01-15 19:04:10.881926", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAc2toRkPyFMTV8MZv8YiRs7dKWOaGqPzO"], "labels": [], "masters": [], "uuids": ["803396ec-d322-45bb-b215-e297daac7584"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:09.841013", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:10.912 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi001.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003032", "end": "2021-01-15 19:04:10.890505", "rc": 0, "start": "2021-01-15 19:04:10.887473", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2021-01-15T19:04:10.951 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:10 +0000 (0:00:00.238) 0:04:21.336 ******** 2021-01-15T19:04:10.982 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:11.017 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:11 +0000 (0:00:00.065) 0:04:21.402 ******** 2021-01-15T19:04:11.043 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-01-15T19:04:11.045 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:11.075 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:11 +0000 (0:00:00.057) 0:04:21.460 ******** 2021-01-15T19:04:11.101 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-01-15T19:04:11.103 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:11.133 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:11 +0000 (0:00:00.057) 0:04:21.518 ******** 2021-01-15T19:04:11.159 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:11.191 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:11 +0000 (0:00:00.057) 0:04:21.575 ******** 2021-01-15T19:04:12.157 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:12.158 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAqgDISbfTflK2x1AfPrL5VhOmpuwLy7zZ'], 'uuids': ['ce99a754-3575-4908-aa3f-38ee72c24cc3'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.068464", "end": "2021-01-15 19:04:12.132771", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAqgDISbfTflK2x1AfPrL5VhOmpuwLy7zZ"], "labels": [], "masters": [], "uuids": ["ce99a754-3575-4908-aa3f-38ee72c24cc3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:11.064307", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:13.392 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAeOZ5bPHYuHBCdSag2lYKh8ekqUr80F1i'], 'uuids': ['bae8ab6d-568c-4de8-ab57-5f163d9fa5c8'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.047178", "end": "2021-01-15 19:04:13.365323", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAeOZ5bPHYuHBCdSag2lYKh8ekqUr80F1i"], "labels": [], "masters": [], "uuids": ["bae8ab6d-568c-4de8-ab57-5f163d9fa5c8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:12.318145", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:13.403 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKJ', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5XTKJ', 'scsi-35000c5009293d5e8', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ', 'wwn-0x5000c5009293d5e8'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XTKJ', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c5009293d5e8', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKJ-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5XTKJ-part1', 'scsi-35000c5009293d5e8-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1', 'wwn-0x5000c5009293d5e8-part1'], 'uuids': ['a313f13c-d1e6-43a8-a40a-989733f55ee1'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKJ", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W5XTKJ", "scsi-35000c5009293d5e8", "scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ", "wwn-0x5000c5009293d5e8"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKJ-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-01-15T19:04:13.405 INFO:teuthology.task.ansible.out:Z1W5XTKJ-part1", "scsi-35000c5009293d5e8-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1", "wwn-0x5000c5009293d5e8-part1"], "labels": [], "masters": [], "uuids": ["a313f13c-d1e6-43a8-a40a-989733f55ee1"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTKJ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293d5e8"}}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:14.657 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAgAA74xTgsSoBDyyMLaWgt4wvNqqk29wt'], 'uuids': ['5425aac5-280e-4ecd-ad40-419203fa178b'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.064671", "end": "2021-01-15 19:04:14.624861", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAgAA74xTgsSoBDyyMLaWgt4wvNqqk29wt"], "labels": [], "masters": [], "uuids": ["5425aac5-280e-4ecd-ad40-419203fa178b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:13.560190", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:14.697 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:14 +0000 (0:00:03.506) 0:04:25.082 ******** 2021-01-15T19:04:15.018 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:15.020 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CApUEvJfzzHbbAvKVy5ZTEJVXX6XlcgjEj'], 'uuids': ['d8d2c0b2-32e6-4f01-8b2e-5c0dd295b85c'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.101394", "end": "2021-01-15 19:04:14.991660", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CApUEvJfzzHbbAvKVy5ZTEJVXX6XlcgjEj"], "labels": [], "masters": [], "uuids": ["d8d2c0b2-32e6-4f01-8b2e-5c0dd295b85c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:14.890266", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0987295 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0987295 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:15.237 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-aAPi1K-INVF-Xq05-weoS-FinR-cP0G-6rfBsr', 'nvme-INTEL_SSDPEDMX020T4_CVPD514100122P0SGN', 'nvme-nvme.8086-43565044353134313030313232503053474e-494e54454c205353445045444d583032305434-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMX020T4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '3907029168', 'sectorsize': '512', 'size': '1.82 TB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037422", "end": "2021-01-15 19:04:15.206829", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-aAPi1K-INVF-Xq05-weoS-FinR-cP0G-6rfBsr", "nvme-INTEL_SSDPEDMX020T4_CVPD514100122P0SGN", "nvme-nvme.8086-43565044353134313030313232503053474e-494e54454c205353445045444d583032305434-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMX020T4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "3907029168", "sectorsize": "512", "size": "1.82 TB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:15.169407", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0348164 s, 3.3 GB/s", "stder 2021-01-15T19:04:15.239 INFO:teuthology.task.ansible.out:r_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0348164 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:15.518 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAc2toRkPyFMTV8MZv8YiRs7dKWOaGqPzO'], 'uuids': ['803396ec-d322-45bb-b215-e297daac7584'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.104673", "end": "2021-01-15 19:04:15.491760", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAc2toRkPyFMTV8MZv8YiRs7dKWOaGqPzO"], "labels": [], "masters": [], "uuids": ["803396ec-d322-45bb-b215-e297daac7584"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:15.387087", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.102068 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.102068 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:15.814 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAqgDISbfTflK2x1AfPrL5VhOmpuwLy7zZ'], 'uuids': ['ce99a754-3575-4908-aa3f-38ee72c24cc3'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.107377", "end": "2021-01-15 19:04:15.791596", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAqgDISbfTflK2x1AfPrL5VhOmpuwLy7zZ"], "labels": [], "masters": [], "uuids": ["ce99a754-3575-4908-aa3f-38ee72c24cc3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:15.684219", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.104683 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.104683 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:16.094 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAeOZ5bPHYuHBCdSag2lYKh8ekqUr80F1i'], 'uuids': ['bae8ab6d-568c-4de8-ab57-5f163d9fa5c8'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.105586", "end": "2021-01-15 19:04:16.071370", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAeOZ5bPHYuHBCdSag2lYKh8ekqUr80F1i"], "labels": [], "masters": [], "uuids": ["bae8ab6d-568c-4de8-ab57-5f163d9fa5c8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:15.965784", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.102919 s, 1.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.102919 s, 1.1 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:16.106 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKJ', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5XTKJ', 'scsi-35000c5009293d5e8', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ', 'wwn-0x5000c5009293d5e8'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5XTKJ', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c5009293d5e8', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XTKJ-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5XTKJ-part1', 'scsi-35000c5009293d5e8-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1', 'wwn-0x5000c5009293d5e8-part1'], 'uuids': ['a313f13c-d1e6-43a8-a40a-989733f55ee1'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKJ", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W5XTKJ", "scsi-35000c5009293d5e8", "scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ", "wwn-0x5000c5009293d5e8"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTKJ-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-01-15T19:04:16.107 INFO:teuthology.task.ansible.out:Z1W5XTKJ-part1", "scsi-35000c5009293d5e8-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W5XTKJ-part1", "wwn-0x5000c5009293d5e8-part1"], "labels": [], "masters": [], "uuids": ["a313f13c-d1e6-43a8-a40a-989733f55ee1"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTKJ", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293d5e8"}}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:16.366 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-01-15T19:04:16.368 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper", "Installed: device-mapper-event-8:1.02.171-5.el8.x86_64", "Installed: lvm2-8:2.03.09-5.el8.x86_64", "Installed: device-mapper-event-libs-8:1.02.171-5.el8.x86_64", "Installed: lvm2-libs-8:2.03.09-5.el8.x86_64", "Installed: device-mapper-libs-8:1.02.171-5.el8.x86_64", "Installed: device-mapper-8:1.02.171-5.el8.x86_64", "Removed: device-mapper-8:1.02.169-3.el8.x86_64", "Removed: device-mapper-event-8:1.02.169-3.el8.x86_64", "Removed: device-mapper-event-libs-8:1.02.169-3.el8.x86_64", "Removed: device-mapper-libs-8:1.02.169-3.el8.x86_64", "Removed: lvm2-8:2.03.08-3.el8.x86_64", "Removed: lvm2-libs-8:2.03.08-3.el8.x86_64"]} 2021-01-15T19:04:16.399 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:01.702) 0:04:26.784 ******** 2021-01-15T19:04:16.406 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:16.407 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAgAA74xTgsSoBDyyMLaWgt4wvNqqk29wt'], 'uuids': ['5425aac5-280e-4ecd-ad40-419203fa178b'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.100655", "end": "2021-01-15 19:04:16.353139", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-3MqhrD3Fi5yAv1EQDxsOqiCpY9Oac5CAgAA74xTgsSoBDyyMLaWgt4wvNqqk29wt"], "labels": [], "masters": [], "uuids": ["5425aac5-280e-4ecd-ad40-419203fa178b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:16.252484", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.098078 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.098078 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:16.449 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:00.049) 0:04:26.834 ******** 2021-01-15T19:04:16.465 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-15T19:04:16.467 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930356359168, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227137783, 'block_used': 12957325, 'inode_total': 61054976, 'inode_available': 60986357, 'inode_used': 68619, 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227137783, "block_size": 4096, "block_total": 240095108, "block_used": 12957325, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986357, "inode_total": 61054976, "inode_used": 68619, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930356359168, "size_total": 983429562368, "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}} 2021-01-15T19:04:16.501 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:00.052) 0:04:26.886 ******** 2021-01-15T19:04:16.689 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi136.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.050098", "end": "2021-01-15 19:04:16.663494", "rc": 0, "start": "2021-01-15 19:04:16.613396", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:16.724 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:00.223) 0:04:27.109 ******** 2021-01-15T19:04:16.730 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-01-15T19:04:16.732 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005560", "end": "2021-01-15 19:04:16.691596", "rc": 0, "start": "2021-01-15 19:04:16.686036", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:16.766 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:00.041) 0:04:27.151 ******** 2021-01-15T19:04:16.804 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-01-15T19:04:16.806 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930356359168, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227137783, 'block_used': 12957325, 'inode_total': 61054976, 'inode_available': 60986357, 'inode_used': 68619, 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227137783, "block_size": 4096, "block_total": 240095108, "block_used": 12957325, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986357, "inode_total": 61054976, "inode_used": 68619, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930356359168, "size_total": 983429562368, "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:16.839 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:00.072) 0:04:27.223 ******** 2021-01-15T19:04:16.876 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-01-15T19:04:16.920 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:16 +0000 (0:00:00.081) 0:04:27.305 ******** 2021-01-15T19:04:16.969 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-01-15T19:04:16.971 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.047440", "end": "2021-01-15 19:04:16.948275", "rc": 0, "start": "2021-01-15 19:04:16.900835", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:17.005 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.085) 0:04:27.390 ******** 2021-01-15T19:04:17.033 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-01-15T19:04:17.068 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.062) 0:04:27.453 ******** 2021-01-15T19:04:17.094 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-01-15T19:04:17.096 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.131 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.063) 0:04:27.516 ******** 2021-01-15T19:04:17.159 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-01-15T19:04:17.160 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.193 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.061) 0:04:27.578 ******** 2021-01-15T19:04:17.220 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-15T19:04:17.222 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.256 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.063) 0:04:27.641 ******** 2021-01-15T19:04:17.282 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-01-15T19:04:17.284 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.316 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.059) 0:04:27.701 ******** 2021-01-15T19:04:17.497 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-01-15T19:04:17.499 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T19:04:17.531 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.215) 0:04:27.916 ******** 2021-01-15T19:04:17.568 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-15T19:04:17.569 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930356453376, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227137806, 'block_used': 12957302, 'inode_total': 61054976, 'inode_available': 60986349, 'inode_used': 68627, 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227137806, "block_size": 4096, "block_total": 240095108, "block_used": 12957302, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986349, "inode_total": 61054976, "inode_used": 68627, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930356453376, "size_total": 983429562368, "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.602 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.070) 0:04:27.986 ******** 2021-01-15T19:04:17.628 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-01-15T19:04:17.630 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.660 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.058) 0:04:28.045 ******** 2021-01-15T19:04:17.696 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:17.699 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.733 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.072) 0:04:28.118 ******** 2021-01-15T19:04:17.759 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:17.761 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.792 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.058) 0:04:28.177 ******** 2021-01-15T19:04:17.819 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-01-15T19:04:17.820 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.852 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.060) 0:04:28.237 ******** 2021-01-15T19:04:17.878 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-01-15T19:04:17.881 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.912 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.059) 0:04:28.297 ******** 2021-01-15T19:04:17.949 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-01-15T19:04:17.951 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:17.983 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:17 +0000 (0:00:00.071) 0:04:28.368 ******** 2021-01-15T19:04:18.685 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-01-15T19:04:18.720 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:18 +0000 (0:00:00.736) 0:04:29.105 ******** 2021-01-15T19:04:18.769 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:18.771 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYfoqZ0o0rQ3Vk6fVWa14OlbYTAk5TAvi'], 'uuids': ['af7f8d0b-9f88-44e5-848f-ba0e37456c80'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.617265", "end": "2021-01-15 19:04:18.732143", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYfoqZ0o0rQ3Vk6fVWa14OlbYTAk5TAvi"], "labels": [], "masters": [], "uuids": ["af7f8d0b-9f88-44e5-848f-ba0e37456c80"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:17.114878", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:19.351 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-01-15T19:04:19.353 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:19.675 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:20.007 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:20.029 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:20.030 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-Ce3OWU-3Ut7-yka0-cjhC-Rtt9-TfYr-1EdBpx', 'nvme-INTEL_SSDPED1D480GA_PHMB7513000D480DGN', 'nvme-nvme.8086-50484d42373531333030304434383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPED1D480GA', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '937703088', 'sectorsize': '512', 'size': '447.13 GB', 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.060327", "end": "2021-01-15 19:04:19.994999", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-Ce3OWU-3Ut7-yka0-cjhC-Rtt9-TfYr-1EdBpx", "nvme-INTEL_SSDPED1D480GA_PHMB7513000D480DGN", "nvme-nvme.8086-50484d42373531333030304434383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:18.934672", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now pa 2021-01-15T19:04:20.032 INFO:teuthology.task.ansible.out:rtition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:20.330 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-01-15T19:04:20.332 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:20.644 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:20.681 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:20 +0000 (0:00:01.961) 0:04:31.066 ******** 2021-01-15T19:04:20.874 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-15T19:04:20.877 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-01-15T19:04:20.909 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:20 +0000 (0:00:00.227) 0:04:31.294 ******** 2021-01-15T19:04:21.113 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-01-15T19:04:21.114 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:21.280 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:21.282 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGuiYAO57ZHXgKjyW8FrCrk5rkDSI1bwW1'], 'uuids': ['670544b3-5e02-46b8-bceb-756e5118ecdd'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.050992", "end": "2021-01-15 19:04:21.246533", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGuiYAO57ZHXgKjyW8FrCrk5rkDSI1bwW1"], "labels": [], "masters": [], "uuids": ["670544b3-5e02-46b8-bceb-756e5118ecdd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:20.195541", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:21.292 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-01-15T19:04:21.294 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:21.470 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:21.650 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:21.661 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:21.698 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:21 +0000 (0:00:00.788) 0:04:32.082 ******** 2021-01-15T19:04:21.725 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-01-15T19:04:21.726 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:21.759 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:21 +0000 (0:00:00.061) 0:04:32.143 ******** 2021-01-15T19:04:21.785 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-01-15T19:04:21.787 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:21.819 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:21 +0000 (0:00:00.060) 0:04:32.204 ******** 2021-01-15T19:04:22.012 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-01-15T19:04:22.014 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:22.049 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:22 +0000 (0:00:00.230) 0:04:32.434 ******** 2021-01-15T19:04:22.087 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-01-15T19:04:22.088 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-01-15T19:04:22.123 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:22 +0000 (0:00:00.073) 0:04:32.507 ******** 2021-01-15T19:04:22.536 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGSQJmpRfp1rgBt90OamY9W8WcVDdjW4MQ'], 'uuids': ['5e9e1853-df6d-4e95-bd5e-5ea6a32bdef0'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.060874", "end": "2021-01-15 19:04:22.504716", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGSQJmpRfp1rgBt90OamY9W8WcVDdjW4MQ"], "labels": [], "masters": [], "uuids": ["5e9e1853-df6d-4e95-bd5e-5ea6a32bdef0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:21.443842", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2021-01-15T19:04:22.537 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2021-01-15T19:04:23.640 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi136.front.sepia.ceph.com] => {"changed": true} 2021-01-15T19:04:23.676 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:23 +0000 (0:00:01.552) 0:04:34.060 ******** 2021-01-15T19:04:23.775 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-01-15T19:04:23.777 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYDaXay403m6kbw2AqhesdLrdYCB3oGX3'], 'uuids': ['4258c5ee-4fc1-49e1-a56d-93933b6db7a0'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.045156", "end": "2021-01-15 19:04:23.746719", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYDaXay403m6kbw2AqhesdLrdYCB3oGX3"], "labels": [], "masters": [], "uuids": ["4258c5ee-4fc1-49e1-a56d-93933b6db7a0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:22.701563", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-01-15T19:04:23.788 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQQD', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4HQQD', 'scsi-35000c500871a6c4f', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD', 'wwn-0x5000c500871a6c4f'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HQQD', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a6c4f', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQQD-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4HQQD-part1', 'scsi-35000c500871a6c4f-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD-part1', 'wwn-0x5000c500871a6c4f-part1'], 'uuids': ['a313f13c-d1e6-43a8-a40a-989733f55ee1'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQQD", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W4HQQD", "scsi-35000c500871a6c4f", "scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD", "wwn-0x5000c500871a6c4f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQQD-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-01-15T19:04:23.790 INFO:teuthology.task.ansible.out:Z1W4HQQD-part1", "scsi-35000c500871a6c4f-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD-part1", "wwn-0x5000c500871a6c4f-part1"], "labels": [], "masters": [], "uuids": ["a313f13c-d1e6-43a8-a40a-989733f55ee1"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQQD", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a6c4f"}}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.248 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-01-15T19:04:24.250 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-01-15T19:04:24.282 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.605) 0:04:34.666 ******** 2021-01-15T19:04:24.308 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-01-15T19:04:24.310 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.342 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.060) 0:04:34.727 ******** 2021-01-15T19:04:24.368 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-01-15T19:04:24.370 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.402 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.060) 0:04:34.787 ******** 2021-01-15T19:04:24.429 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-01-15T19:04:24.430 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.462 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.059) 0:04:34.847 ******** 2021-01-15T19:04:24.488 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-01-15T19:04:24.490 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.521 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.059) 0:04:34.906 ******** 2021-01-15T19:04:24.547 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-01-15T19:04:24.549 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.602 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.080) 0:04:34.986 ******** 2021-01-15T19:04:24.628 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-01-15T19:04:24.629 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.663 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.061) 0:04:35.048 ******** 2021-01-15T19:04:24.690 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.728 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.064) 0:04:35.113 ******** 2021-01-15T19:04:24.754 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-01-15T19:04:24.756 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.792 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.064) 0:04:35.177 ******** 2021-01-15T19:04:24.818 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-01-15T19:04:24.820 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.853 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.060) 0:04:35.238 ******** 2021-01-15T19:04:24.879 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-01-15T19:04:24.881 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.914 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.061) 0:04:35.299 ******** 2021-01-15T19:04:24.941 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-01-15T19:04:24.943 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:24.975 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:24 +0000 (0:00:00.060) 0:04:35.360 ******** 2021-01-15T19:04:25.002 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-01-15T19:04:25.004 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:25.038 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:25 +0000 (0:00:00.063) 0:04:35.423 ******** 2021-01-15T19:04:25.065 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:25.067 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGlEUNycNKeQqvvG0KHTXzbEUTjCBmbgPl'], 'uuids': ['501f7cf6-449e-43ae-b404-e80e07fa8f4f'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.083032", "end": "2021-01-15 19:04:25.040209", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGlEUNycNKeQqvvG0KHTXzbEUTjCBmbgPl"], "labels": [], "masters": [], "uuids": ["501f7cf6-449e-43ae-b404-e80e07fa8f4f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:23.957177", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2021-01-15T19:04:25.068 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2021-01-15T19:04:25.101 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:25 +0000 (0:00:00.062) 0:04:35.486 ******** 2021-01-15T19:04:25.148 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:25 +0000 (0:00:00.047) 0:04:35.533 ******** 2021-01-15T19:04:25.154 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-01-15T19:04:25.156 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:25.191 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:25 +0000 (0:00:00.042) 0:04:35.576 ******** 2021-01-15T19:04:25.217 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-01-15T19:04:25.219 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:25.251 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:25 +0000 (0:00:00.060) 0:04:35.636 ******** 2021-01-15T19:04:25.477 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:25.479 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYfoqZ0o0rQ3Vk6fVWa14OlbYTAk5TAvi'], 'uuids': ['af7f8d0b-9f88-44e5-848f-ba0e37456c80'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.096435", "end": "2021-01-15 19:04:25.444447", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYfoqZ0o0rQ3Vk6fVWa14OlbYTAk5TAvi"], "labels": [], "masters": [], "uuids": ["af7f8d0b-9f88-44e5-848f-ba0e37456c80"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:25.348012", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0934631 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0934631 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:25.698 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-Ce3OWU-3Ut7-yka0-cjhC-Rtt9-TfYr-1EdBpx', 'nvme-INTEL_SSDPED1D480GA_PHMB7513000D480DGN', 'nvme-nvme.8086-50484d42373531333030304434383044474e-494e54454c2053534450454431443438304741-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPED1D480GA', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '937703088', 'sectorsize': '512', 'size': '447.13 GB', 'host': 'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.037146", "end": "2021-01-15 19:04:25.665359", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-Ce3OWU-3Ut7-yka0-cjhC-Rtt9-TfYr-1EdBpx", "nvme-INTEL_SSDPED1D480GA_PHMB7513000D480DGN", "nvme-nvme.8086-50484d42373531333030304434383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:25.628213", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.034277 s, 3.4 GB/s", "stderr_lines": ["1 2021-01-15T19:04:25.700 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.034277 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:25.724 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-01-15T19:04:25.725 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2021-01-15T19:04:25.759 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:25 +0000 (0:00:00.505) 0:04:36.142 ******** 2021-01-15T19:04:25.984 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:25.986 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGuiYAO57ZHXgKjyW8FrCrk5rkDSI1bwW1'], 'uuids': ['670544b3-5e02-46b8-bceb-756e5118ecdd'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.094808", "end": "2021-01-15 19:04:25.951087", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGuiYAO57ZHXgKjyW8FrCrk5rkDSI1bwW1"], "labels": [], "masters": [], "uuids": ["670544b3-5e02-46b8-bceb-756e5118ecdd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:25.856279", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0920786 s, 1.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0920786 s, 1.3 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:26.080 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-01-15T19:04:26.082 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2021-01-15 18:59:33 UTC", "ActiveEnterTimestampMonotonic": "13114348", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target -.mount ntpd.service systemd-tmpfiles-setup.service sysinit.target systemd-journald.socket ntpdate.service sntp.service tmp.mount system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2021-01-15 18:59:32 UTC", "AssertTimestampMonotonic": "11918501", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2021-01-15 18:59:32 UTC", "ConditionTimestampMonotonic": "11918453", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0 2021-01-15T19:04:26.084 INFO:teuthology.task.ansible.out:", "DefaultDependencies": "yes", "Delegate": "no", "Description": "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "891", "ExecMainStartTimestamp": "Fri 2021-01-15 18:59:33 UTC", "ExecMainStartTimestampMonotonic": "13032477", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2021-01-15 18:59:32 UTC", "InactiveExitTimestampMonotonic": "11932776", "InvocationID": "9364df6b6daa45d882fbf8834e3038db", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity 2021-01-15T19:04:26.085 INFO:teuthology.task.ansible.out:", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127261", "LimitNPROCSoft": "127261", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127261", "LimitSIGPENDINGSoft": "127261", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "891", "MemoryAccounting": "yes", "MemoryCurrent": "3452928", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOn 2021-01-15T19:04:26.087 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2021-01-15 18:59:33 UTC", "StateChangeTimestampMonotonic": "13114348", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203618", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2021-01-15 18:59:33 UTC", "WatchdogTimestampMonotonic": "13032491", "WatchdogUSec": "0"}} 2021-01-15T19:04:26.115 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.358) 0:04:36.500 ******** 2021-01-15T19:04:26.142 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:26.144 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-01-15T19:04:26.175 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.059) 0:04:36.560 ******** 2021-01-15T19:04:26.212 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:26.214 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-01-15T19:04:26.250 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.074) 0:04:36.635 ******** 2021-01-15T19:04:26.267 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:26.269 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGSQJmpRfp1rgBt90OamY9W8WcVDdjW4MQ'], 'uuids': ['5e9e1853-df6d-4e95-bd5e-5ea6a32bdef0'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.096210", "end": "2021-01-15 19:04:26.238018", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGSQJmpRfp1rgBt90OamY9W8WcVDdjW4MQ"], "labels": [], "masters": [], "uuids": ["5e9e1853-df6d-4e95-bd5e-5ea6a32bdef0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:26.141808", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0935791 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0935791 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:26.279 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:26.314 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.063) 0:04:36.698 ******** 2021-01-15T19:04:26.340 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-01-15T19:04:26.342 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:26.374 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.060) 0:04:36.759 ******** 2021-01-15T19:04:26.400 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-01-15T19:04:26.402 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:26.435 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.060) 0:04:36.819 ******** 2021-01-15T19:04:26.545 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:26.546 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYDaXay403m6kbw2AqhesdLrdYCB3oGX3'], 'uuids': ['4258c5ee-4fc1-49e1-a56d-93933b6db7a0'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.095201", "end": "2021-01-15 19:04:26.519164", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGYDaXay403m6kbw2AqhesdLrdYCB3oGX3"], "labels": [], "masters": [], "uuids": ["4258c5ee-4fc1-49e1-a56d-93933b6db7a0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:26.423963", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0924587 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0924587 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:26.558 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQQD', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4HQQD', 'scsi-35000c500871a6c4f', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD', 'wwn-0x5000c500871a6c4f'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4HQQD', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871a6c4f', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4HQQD-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4HQQD-part1', 'scsi-35000c500871a6c4f-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD-part1', 'wwn-0x5000c500871a6c4f-part1'], 'uuids': ['a313f13c-d1e6-43a8-a40a-989733f55ee1'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQQD", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W4HQQD", "scsi-35000c500871a6c4f", "scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD", "wwn-0x5000c500871a6c4f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQQD-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4HQQD-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-01-15T19:04:26.560 INFO:teuthology.task.ansible.out:Z1W4HQQD-part1", "scsi-35000c500871a6c4f-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W4HQQD-part1", "wwn-0x5000c500871a6c4f-part1"], "labels": [], "masters": [], "uuids": ["a313f13c-d1e6-43a8-a40a-989733f55ee1"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQQD", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a6c4f"}}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:26.632 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-01-15T19:04:26.634 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2021-01-15T19:04:26.669 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:26 +0000 (0:00:00.234) 0:04:37.054 ******** 2021-01-15T19:04:26.832 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-01-15T19:04:26.895 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGlEUNycNKeQqvvG0KHTXzbEUTjCBmbgPl'], 'uuids': ['501f7cf6-449e-43ae-b404-e80e07fa8f4f'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.095860", "end": "2021-01-15 19:04:26.809322", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-foOXdnI2TUN8x4xdj4hKhjj11zKxU1jGlEUNycNKeQqvvG0KHTXzbEUTjCBmbgPl"], "labels": [], "masters": [], "uuids": ["501f7cf6-449e-43ae-b404-e80e07fa8f4f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-01-15 19:04:26.713462", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0931262 s, 1.2 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0931262 s, 1.2 GB/s"], "stdout": "", "stdout_lines": []} Friday 15 January 2021 19:04:26 +0000 (0:00:00.207) 0:04:37.262 ******** 2021-01-15T19:04:27.147 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-01-15T19:04:27.150 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.051758", "end": "2021-01-15 19:04:27.116889", "rc": 0, "start": "2021-01-15 19:04:27.065131", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:27.181 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.304) 0:04:37.566 ******** 2021-01-15T19:04:27.223 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-01-15T19:04:27.225 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2021-01-15T19:04:27.259 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.076) 0:04:37.643 ******** 2021-01-15T19:04:27.432 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-01-15T19:04:27.434 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.041894", "end": "2021-01-15 19:04:27.403392", "rc": 0, "start": "2021-01-15 19:04:27.361498", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:27.481 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.221) 0:04:37.864 ******** 2021-01-15T19:04:27.507 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-01-15T19:04:27.546 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.066) 0:04:37.931 ******** 2021-01-15T19:04:27.575 INFO:teuthology.task.ansible.out: 2021-01-15T19:04:27.578 INFO:teuthology.task.ansible.out:TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:27.639 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.092) 0:04:38.024 ******** 2021-01-15T19:04:27.677 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-01-15T19:04:27.678 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:27.713 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.074) 0:04:38.098 ******** 2021-01-15T19:04:27.748 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-15T19:04:27.750 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:27.784 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.070) 0:04:38.169 ******** 2021-01-15T19:04:27.821 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-01-15T19:04:27.823 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:27.855 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:27 +0000 (0:00:00.070) 0:04:38.240 ******** 2021-01-15T19:04:28.071 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-01-15T19:04:28.075 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-01-15T19:04:28.119 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.264) 0:04:38.504 ******** 2021-01-15T19:04:28.176 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-01-15T19:04:28.178 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930356359168, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227137783, 'block_used': 12957325, 'inode_total': 61054976, 'inode_available': 60986357, 'inode_used': 68619, 'uuid': 'a313f13c-d1e6-43a8-a40a-989733f55ee1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227137783, "block_size": 4096, "block_total": 240095108, "block_used": 12957325, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986357, "inode_total": 61054976, "inode_used": 68619, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930356359168, "size_total": 983429562368, "uuid": "a313f13c-d1e6-43a8-a40a-989733f55ee1"}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.217 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.098) 0:04:38.602 ******** 2021-01-15T19:04:28.259 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-01-15T19:04:28.261 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.305 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.088) 0:04:38.690 ******** 2021-01-15T19:04:28.336 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:28.338 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.371 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.065) 0:04:38.756 ******** 2021-01-15T19:04:28.410 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:28.412 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.443 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.071) 0:04:38.827 ******** 2021-01-15T19:04:28.473 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-01-15T19:04:28.475 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.514 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.071) 0:04:38.899 ******** 2021-01-15T19:04:28.544 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.579 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.064) 0:04:38.964 ******** 2021-01-15T19:04:28.606 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-01-15T19:04:28.608 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages via pip] ************************************* ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-01-15T19:04:28.646 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.067) 0:04:39.031 ******** 2021-01-15T19:04:28.689 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.042) 0:04:39.074 ******** 2021-01-15T19:04:28.717 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-01-15T19:04:28.719 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:28.753 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.064) 0:04:39.138 ******** 2021-01-15T19:04:28.780 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-01-15T19:04:28.782 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-01-15T19:04:28.817 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:28 +0000 (0:00:00.063) 0:04:39.202 ******** 2021-01-15T19:04:29.000 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-01-15T19:04:29.002 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-01-15T19:04:29.041 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.223) 0:04:39.425 ******** 2021-01-15T19:04:29.067 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-01-15T19:04:29.069 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.104 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.062) 0:04:39.488 ******** 2021-01-15T19:04:29.109 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-01-15T19:04:29.111 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-01-15T19:04:29.148 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.044) 0:04:39.533 ******** 2021-01-15T19:04:29.155 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-01-15T19:04:29.156 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-01-15T19:04:29.192 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.044) 0:04:39.577 ******** 2021-01-15T19:04:29.219 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-01-15T19:04:29.221 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.275 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.082) 0:04:39.660 ******** 2021-01-15T19:04:29.472 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-01-15T19:04:29.474 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-01-15T19:04:29.527 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.251) 0:04:39.912 ******** 2021-01-15T19:04:29.540 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-01-15T19:04:29.542 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:29.720 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-01-15T19:04:29.722 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-01-15T19:04:29.755 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.226) 0:04:40.139 ******** 2021-01-15T19:04:29.781 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-01-15T19:04:29.783 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.816 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.061) 0:04:40.200 ******** 2021-01-15T19:04:29.852 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-01-15T19:04:29.853 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.863 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.864 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.865 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi136.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.900 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.083) 0:04:40.284 ******** 2021-01-15T19:04:29.938 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-01-15T19:04:29.940 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.950 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.952 INFO:teuthology.task.ansible.out: skipping: [smithi136.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi136.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:29.954 INFO:teuthology.task.ansible.out: 2021-01-15T19:04:29.987 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:29 +0000 (0:00:00.087) 0:04:40.372 ******** 2021-01-15T19:04:30.014 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:30.015 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.050 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.062) 0:04:40.435 ******** 2021-01-15T19:04:30.077 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.110 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.060) 0:04:40.495 ******** 2021-01-15T19:04:30.139 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-01-15T19:04:30.141 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.175 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.064) 0:04:40.560 ******** 2021-01-15T19:04:30.196 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-01-15T19:04:30.197 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:30.207 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-01-15T19:04:30.209 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.241 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.065) 0:04:40.625 ******** 2021-01-15T19:04:30.267 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-01-15T19:04:30.269 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.302 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.061) 0:04:40.687 ******** 2021-01-15T19:04:30.329 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-15T19:04:30.331 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.368 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.065) 0:04:40.753 ******** 2021-01-15T19:04:30.394 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-01-15T19:04:30.396 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.430 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.062) 0:04:40.815 ******** 2021-01-15T19:04:30.457 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-15T19:04:30.459 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:30.512 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.081) 0:04:40.897 ******** 2021-01-15T19:04:30.532 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-01-15T19:04:30.533 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:30.857 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-01-15T19:04:30.894 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:30 +0000 (0:00:00.381) 0:04:41.279 ******** 2021-01-15T19:04:31.100 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-01-15T19:04:31.102 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-01-15T19:04:31.136 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:31 +0000 (0:00:00.241) 0:04:41.521 ******** 2021-01-15T19:04:31.362 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-01-15T19:04:31.364 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:31.550 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:31.736 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-15T19:04:31.738 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.006047", "end": "2021-01-15 19:04:31.694781", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-01-15 19:04:30.688734", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:31.916 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-01-15T19:04:31.918 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-15T19:04:31.918 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003036", "end": "2021-01-15 19:04:31.886467", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-01-15 19:04:31.883431", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:31.940 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** skipping: [smithi001.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:31.976 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:31 +0000 (0:00:00.840) 0:04:42.361 ******** 2021-01-15T19:04:32.015 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-01-15T19:04:32.016 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:32.060 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.083) 0:04:42.444 ******** 2021-01-15T19:04:32.088 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-15T19:04:32.090 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002865", "end": "2021-01-15 19:04:32.058830", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-01-15 19:04:32.055965", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:32.103 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-01-15T19:04:32.105 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:32.142 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.081) 0:04:42.527 ******** 2021-01-15T19:04:32.263 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-15T19:04:32.265 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.003145", "end": "2021-01-15 19:04:32.236583", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-01-15 19:04:32.233438", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:32.308 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.166) 0:04:42.693 ******** 2021-01-15T19:04:32.337 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-01-15T19:04:32.370 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.062) 0:04:42.755 ******** 2021-01-15T19:04:32.377 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-01-15T19:04:32.378 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:32.421 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.050) 0:04:42.806 ******** 2021-01-15T19:04:32.460 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-01-15T19:04:32.461 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-01-15T19:04:32.500 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.079) 0:04:42.885 ******** 2021-01-15T19:04:32.561 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-01-15T19:04:32.563 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2021-01-15T19:04:32.722 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi001.front.sepia.ceph.com] => {"changed": true} 2021-01-15T19:04:32.755 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:32 +0000 (0:00:00.254) 0:04:43.140 ******** 2021-01-15T19:04:33.064 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-01-15T19:04:33.066 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-01-15T19:04:33.100 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.344) 0:04:43.484 ******** 2021-01-15T19:04:33.126 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-01-15T19:04:33.128 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.161 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.061) 0:04:43.546 ******** 2021-01-15T19:04:33.188 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-01-15T19:04:33.189 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.221 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.059) 0:04:43.606 ******** 2021-01-15T19:04:33.249 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-01-15T19:04:33.250 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.283 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.061) 0:04:43.668 ******** 2021-01-15T19:04:33.310 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-01-15T19:04:33.313 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.345 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.062) 0:04:43.730 ******** 2021-01-15T19:04:33.372 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-01-15T19:04:33.374 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.405 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.060) 0:04:43.790 ******** 2021-01-15T19:04:33.437 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-01-15T19:04:33.439 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.472 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.066) 0:04:43.857 ******** 2021-01-15T19:04:33.499 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-01-15T19:04:33.501 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.533 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.061) 0:04:43.918 ******** 2021-01-15T19:04:33.560 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-01-15T19:04:33.562 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.595 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.061) 0:04:43.979 ******** 2021-01-15T19:04:33.621 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-01-15T19:04:33.623 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.654 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.059) 0:04:44.039 ******** 2021-01-15T19:04:33.681 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.716 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.061) 0:04:44.101 ******** 2021-01-15T19:04:33.743 INFO:teuthology.task.ansible.out: 2021-01-15T19:04:33.745 INFO:teuthology.task.ansible.out:TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.791 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.074) 0:04:44.175 ******** 2021-01-15T19:04:33.819 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.852 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.061) 0:04:44.237 ******** 2021-01-15T19:04:33.882 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-01-15T19:04:33.883 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:33.935 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:33 +0000 (0:00:00.082) 0:04:44.320 ******** 2021-01-15T19:04:33.966 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-01-15T19:04:33.968 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:34.010 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:34 +0000 (0:00:00.074) 0:04:44.395 ******** 2021-01-15T19:04:34.038 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-01-15T19:04:34.040 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:34.072 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:34 +0000 (0:00:00.061) 0:04:44.457 ******** 2021-01-15T19:04:34.560 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-01-15T19:04:34.562 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2021-01-15T19:04:34.595 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:34 +0000 (0:00:00.522) 0:04:44.980 ******** 2021-01-15T19:04:34.917 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-01-15T19:04:34.919 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2021-01-15 18:59:22 UTC", "ActiveEnterTimestampMonotonic": "16370179", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sntp.service basic.target systemd-journald.socket ntpdate.service ntpd.service -.mount systemd-tmpfiles-setup.service tmp.mount system.slice sysinit.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2021-01-15 18:59:21 UTC", "AssertTimestampMonotonic": "14921314", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2021-01-15 18:59:21 UTC", "ConditionTimestampMonotonic": "14921245", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service ntpd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0 2021-01-15T19:04:34.920 INFO:teuthology.task.ansible.out:", "DefaultDependencies": "yes", "Delegate": "no", "Description": "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "923", "ExecMainStartTimestamp": "Fri 2021-01-15 18:59:22 UTC", "ExecMainStartTimestampMonotonic": "16262114", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2021-01-15 18:59:21 UTC", "InactiveExitTimestampMonotonic": "14927902", "InvocationID": "cf37f6289f8944238a9ae64ca3d24d0f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity 2021-01-15T19:04:34.922 INFO:teuthology.task.ansible.out:", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127286", "LimitNPROCSoft": "127286", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127286", "LimitSIGPENDINGSoft": "127286", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "923", "MemoryAccounting": "yes", "MemoryCurrent": "3670016", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOn 2021-01-15T19:04:34.923 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2021-01-15 18:59:22 UTC", "StateChangeTimestampMonotonic": "16370179", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203657", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2021-01-15 18:59:22 UTC", "WatchdogTimestampMonotonic": "16262310", "WatchdogUSec": "0"}} 2021-01-15T19:04:34.953 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:34 +0000 (0:00:00.357) 0:04:45.337 ******** 2021-01-15T19:04:34.979 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-01-15T19:04:34.981 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-01-15T19:04:35.014 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:35 +0000 (0:00:00.061) 0:04:45.399 ******** 2021-01-15T19:04:35.052 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-01-15T19:04:35.097 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:35 +0000 (0:00:00.083) 0:04:45.482 ******** 2021-01-15T19:04:35.125 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-01-15T19:04:35.126 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:35.159 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:35 +0000 (0:00:00.061) 0:04:45.544 ******** 2021-01-15T19:04:35.186 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-01-15T19:04:35.188 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:35.221 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:35 +0000 (0:00:00.061) 0:04:45.606 ******** 2021-01-15T19:04:35.249 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-01-15T19:04:35.251 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:35.284 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:35 +0000 (0:00:00.062) 0:04:45.669 ******** 2021-01-15T19:04:35.489 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-01-15T19:04:35.491 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2021-01-15T19:04:35.522 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:35 +0000 (0:00:00.238) 0:04:45.907 ******** 2021-01-15T19:04:36.010 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-01-15T19:04:36.011 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2021-01-15T19:04:36.044 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:36 +0000 (0:00:00.521) 0:04:46.429 ******** 2021-01-15T19:04:37.174 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-01-15T19:04:37.176 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-01-15T19:04:37.208 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:01.163) 0:04:47.592 ******** 2021-01-15T19:04:37.234 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-01-15T19:04:37.235 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:37.268 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.060) 0:04:47.652 ******** 2021-01-15T19:04:37.294 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2021-01-15T19:04:37.328 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.060) 0:04:47.713 ******** 2021-01-15T19:04:37.524 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-01-15T19:04:37.526 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-01-15T19:04:37.560 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.231) 0:04:47.944 ******** 2021-01-15T19:04:37.586 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-01-15T19:04:37.588 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:37.621 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.060) 0:04:48.005 ******** 2021-01-15T19:04:37.648 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-01-15T19:04:37.682 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.061) 0:04:48.066 ******** 2021-01-15T19:04:37.711 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-01-15T19:04:37.714 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:37.745 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.063) 0:04:48.130 ******** 2021-01-15T19:04:37.941 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-01-15T19:04:37.943 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-01-15T19:04:37.976 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:37 +0000 (0:00:00.230) 0:04:48.361 ******** 2021-01-15T19:04:38.171 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-01-15T19:04:38.173 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-01-15T19:04:38.217 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.240) 0:04:48.602 ******** 2021-01-15T19:04:38.244 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-01-15T19:04:38.246 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.295 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.078) 0:04:48.680 ******** 2021-01-15T19:04:38.325 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-01-15T19:04:38.327 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.348 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.350 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.387 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.091) 0:04:48.772 ******** 2021-01-15T19:04:38.425 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-01-15T19:04:38.426 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.437 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.441 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.448 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.492 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.103) 0:04:48.876 ******** 2021-01-15T19:04:38.518 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.554 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.063) 0:04:48.939 ******** 2021-01-15T19:04:38.582 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.618 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.063) 0:04:49.002 ******** 2021-01-15T19:04:38.645 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-01-15T19:04:38.647 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.681 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.063) 0:04:49.066 ******** 2021-01-15T19:04:38.708 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-01-15T19:04:38.710 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.746 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.065) 0:04:49.131 ******** 2021-01-15T19:04:38.773 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-01-15T19:04:38.775 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.813 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.065) 0:04:49.197 ******** 2021-01-15T19:04:38.839 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.887 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.074) 0:04:49.271 ******** 2021-01-15T19:04:38.913 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-01-15T19:04:38.915 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:38.959 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:38 +0000 (0:00:00.072) 0:04:49.344 ******** 2021-01-15T19:04:38.987 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-01-15T19:04:38.989 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:39.025 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:39 +0000 (0:00:00.066) 0:04:49.410 ******** 2021-01-15T19:04:39.252 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-01-15T19:04:39.254 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.005368", "end": "2021-01-15 19:04:39.218718", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-01-15 19:04:39.213350", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:39.434 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003207", "end": "2021-01-15 19:04:39.405559", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-01-15 19:04:39.402352", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:40.642 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:01.004124", "end": "2021-01-15 19:04:40.592133", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-01-15 19:04:39.588009", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:40.852 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.003294", "end": "2021-01-15 19:04:40.812202", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-01-15 19:04:40.808908", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:40.904 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:40 +0000 (0:00:01.878) 0:04:51.288 ******** 2021-01-15T19:04:40.945 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-01-15T19:04:40.991 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:40 +0000 (0:00:00.087) 0:04:51.376 ******** 2021-01-15T19:04:41.204 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2021-01-15T19:04:41.318 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-01-15T19:04:41.348 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:41 +0000 (0:00:00.357) 0:04:51.733 ******** 2021-01-15T19:04:41.380 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:41 +0000 (0:00:00.031) 0:04:51.765 ******** 2021-01-15T19:04:42.016 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi001.front.sepia.ceph.com] 2021-01-15T19:04:42.066 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] 2021-01-15T19:04:42.106 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.725) 0:04:52.490 ******** 2021-01-15T19:04:42.141 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-15T19:04:42.142 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:42.175 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.068) 0:04:52.559 ******** 2021-01-15T19:04:42.209 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.034) 0:04:52.593 ******** 2021-01-15T19:04:42.215 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-15T19:04:42.217 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "yum"}, "changed": false} 2021-01-15T19:04:42.251 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.042) 0:04:52.636 ******** 2021-01-15T19:04:42.257 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-15T19:04:42.259 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:42.297 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.045) 0:04:52.682 ******** 2021-01-15T19:04:42.303 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-01-15T19:04:42.304 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml) => {"ansible_facts": {"container_packages": ["podman", "podman-docker"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"} 2021-01-15T19:04:42.340 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.043) 0:04:52.725 ******** 2021-01-15T19:04:42.346 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-15T19:04:42.348 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "yum"}, "changed": false} 2021-01-15T19:04:42.384 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.043) 0:04:52.769 ******** 2021-01-15T19:04:42.421 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-01-15T19:04:42.423 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml) => {"ansible_facts": {"container_packages": ["podman", "podman-docker"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"} 2021-01-15T19:04:42.458 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:42 +0000 (0:00:00.074) 0:04:52.843 ******** 2021-01-15T19:04:43.435 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-01-15T19:04:43.437 INFO:teuthology.task.ansible.out:ok: [smithi001.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: podman", "Installed: podman-docker"]} 2021-01-15T19:04:43.472 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:43 +0000 (0:00:01.013) 0:04:53.856 ******** 2021-01-15T19:04:43.499 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: podman", "Installed: podman-docker"]} 2021-01-15T19:04:43.501 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [smithi001.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/containers/registries.conf"}, "changed": false} 2021-01-15T19:04:43.535 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:43 +0000 (0:00:00.062) 0:04:53.919 ******** 2021-01-15T19:04:43.568 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:43 +0000 (0:00:00.032) 0:04:53.952 ******** 2021-01-15T19:04:43.573 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-15T19:04:43.575 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:43.612 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:43 +0000 (0:00:00.042) 0:04:53.995 ******** 2021-01-15T19:04:43.617 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-01-15T19:04:43.618 INFO:teuthology.task.ansible.out:ok: [smithi136.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/containers/registries.conf"}, "changed": false} 2021-01-15T19:04:43.656 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:43 +0000 (0:00:00.045) 0:04:54.040 ******** 2021-01-15T19:04:43.683 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:43.718 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:43 +0000 (0:00:00.061) 0:04:54.102 ******** 2021-01-15T19:04:43.829 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-01-15T19:04:43.831 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:43.934 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:44.029 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:44.070 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:44 +0000 (0:00:00.352) 0:04:54.455 ******** 2021-01-15T19:04:44.129 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-01-15T19:04:44.169 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:44 +0000 (0:00:00.099) 0:04:54.554 ******** 2021-01-15T19:04:44.604 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-01-15T19:04:44.606 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "ad23d23287af05eb6dfc6847a1957988954f5ccf", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "97bf7a9f95b52e04e40f96163cd1717c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 2134, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737484.1084096-47737618008827/source", "state": "file", "uid": 0} 2021-01-15T19:04:44.679 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "ad23d23287af05eb6dfc6847a1957988954f5ccf", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "97bf7a9f95b52e04e40f96163cd1717c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:cert_t:s0", "size": 2134, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737484.2084725-142663703282695/source", "state": "file", "uid": 0} 2021-01-15T19:04:45.110 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "ad23d23287af05eb6dfc6847a1957988954f5ccf", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "97bf7a9f95b52e04e40f96163cd1717c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2134, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737484.6082048-180964837506174/source", "state": "file", "uid": 0} 2021-01-15T19:04:45.147 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:45 +0000 (0:00:00.978) 0:04:55.532 ******** 2021-01-15T19:04:45.164 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "ad23d23287af05eb6dfc6847a1957988954f5ccf", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "97bf7a9f95b52e04e40f96163cd1717c", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 2134, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1610737484.686575-112916114973092/source", "state": "file", "uid": 0} 2021-01-15T19:04:45.205 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:45 +0000 (0:00:00.058) 0:04:55.590 ******** 2021-01-15T19:04:48.429 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-01-15T19:04:48.431 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-cbrs6uqz-build\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nInstalling collected packages: toml, docopt, registries-conf-ctl\n Running setup.py install for docopt: started\n Running setup.py install for docopt: finished with status 'done'\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-cbrs6uqz-build", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files. 2021-01-15T19:04:48.433 INFO:teuthology.task.ansible.out:pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Installing collected packages: toml, docopt, registries-conf-ctl", " Running setup.py install for docopt: started", " Running setup.py install for docopt: finished with status 'done'", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-01-15T19:04:48.466 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:48 +0000 (0:00:03.260) 0:04:58.850 ******** 2021-01-15T19:04:48.493 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-01-15T19:04:48.495 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:48.532 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:48 +0000 (0:00:00.066) 0:04:58.917 ******** 2021-01-15T19:04:48.538 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-01-15T19:04:48.540 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-e0d3vq8r-build\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nInstalling collected packages: toml, docopt, registries-conf-ctl\n Running setup.py install for docopt: started\n Running setup.py install for docopt: finished with status 'done'\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-e0d3vq8r-build", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files. 2021-01-15T19:04:48.541 INFO:teuthology.task.ansible.out:pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Installing collected packages: toml, docopt, registries-conf-ctl", " Running setup.py install for docopt: started", " Running setup.py install for docopt: finished with status 'done'", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-01-15T19:04:48.576 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:48 +0000 (0:00:00.043) 0:04:58.961 ******** 2021-01-15T19:04:48.582 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/containers/registries.conf if necessary] **** 2021-01-15T19:04:48.584 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:48.619 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:48 +0000 (0:00:00.043) 0:04:59.004 ******** 2021-01-15T19:04:48.625 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-01-15T19:04:48.627 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:48.663 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:48 +0000 (0:00:00.043) 0:04:59.048 ******** 2021-01-15T19:04:48.700 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/containers/registries.conf if necessary] **** 2021-01-15T19:04:48.702 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:48.736 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:48 +0000 (0:00:00.073) 0:04:59.121 ******** 2021-01-15T19:04:48.971 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-01-15T19:04:48.973 INFO:teuthology.task.ansible.out:changed: [smithi136.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.158543", "end": "2021-01-15 19:04:48.948277", "rc": 0, "start": "2021-01-15 19:04:48.789734", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:49.008 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:49 +0000 (0:00:00.271) 0:04:59.393 ******** 2021-01-15T19:04:49.034 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-01-15T19:04:49.036 INFO:teuthology.task.ansible.out:skipping: [smithi136.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:49.124 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-01-15T19:04:49.125 INFO:teuthology.task.ansible.out:changed: [smithi001.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.184798", "end": "2021-01-15 19:04:49.098608", "rc": 0, "start": "2021-01-15 19:04:48.913810", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-01-15T19:04:49.158 INFO:teuthology.task.ansible.out:Friday 15 January 2021 19:04:49 +0000 (0:00:00.150) 0:04:59.543 ******** 2021-01-15T19:04:49.195 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-01-15T19:04:49.197 INFO:teuthology.task.ansible.out:skipping: [smithi001.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-01-15T19:04:49.293 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-01-15T19:04:49.297 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-01-15T19:04:49.300 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-01-15T19:04:49.302 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi001.front.sepia.ceph.com : ok=126 changed=46 unreachable=0 failed=0 skipped=183 rescued=0 ignored=3 smithi136.front.sepia.ceph.com : ok=126 changed=46 unreachable=0 failed=0 skipped=183 rescued=0 ignored=3 Friday 15 January 2021 19:04:49 +0000 (0:00:00.142) 0:04:59.685 ******** =============================================================================== testnode : Install packages -------------------------------------------- 33.53s common : nrpe - Ensure SELinux policy is up to date -------------------- 33.38s users : Create all admin users with sudo access. ----------------------- 31.29s users : Update authorized_keys using the keys repo --------------------- 28.47s common : nrpe - Remove SELinux policy package -------------------------- 10.22s common : nrpe - Load SELinux policy package ----------------------------- 9.53s users : Remove revoked users -------------------------------------------- 7.68s testnode : Make sure rpm dependencies are installed --------------------- 3.51s common : nrpe - Install SELinux dependencies ---------------------------- 3.35s container-host : Install registries-conf-ctl ---------------------------- 3.26s common : Install Kerberos Packages (RedHat) ----------------------------- 3.12s testnode : Ensure ceph packages are not present. ------------------------ 2.59s testnode : Ensure ceph dependency packages are not present. ------------- 2.22s common : Set the hardware clock ----------------------------------------- 2.16s users : Update authorized_keys for each user with literal keys ---------- 1.91s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.88s Gathering Facts --------------------------------------------------------- 1.67s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.42s testnode : Remove packages ------------------- 2021-01-15T19:04:49.303 INFO:teuthology.task.ansible.out:--------------------------- 1.22s testnode : Install packages via pip ------------------------------------- 1.16s 2021-01-15T19:04:49.613 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2021-01-15T19:04:49.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T19:04:49.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T19:04:49.687 DEBUG:teuthology.orchestra.run.smithi001:> true 2021-01-15T19:04:49.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-15T19:04:49.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-15T19:04:49.858 DEBUG:teuthology.orchestra.run.smithi136:> true 2021-01-15T19:04:49.950 INFO:teuthology.run_tasks:Running task clock... 2021-01-15T19:04:49.964 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-01-15T19:04:49.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-01-15T19:04:49.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-01-15T19:04:50.000 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2021-01-15T19:04:50.017 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2021-01-15T19:04:50.052 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2021-01-15T19:04:50.063 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2021-01-15T19:04:50.079 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2021-01-15T19:04:50.095 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2021-01-15T19:04:50.156 INFO:teuthology.orchestra.run.smithi001.stderr:bash: ntpq: command not found 2021-01-15T19:04:50.159 INFO:teuthology.orchestra.run.smithi001.stdout:210 Number of sources = 4 2021-01-15T19:04:50.160 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2021-01-15T19:04:50.160 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2021-01-15T19:04:50.160 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.160 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.160 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.161 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.161 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-01-15T19:04:50.161 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-01-15T19:04:50.196 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2021-01-15T19:04:50.212 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2021-01-15T19:04:50.256 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2021-01-15T19:04:50.267 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2021-01-15T19:04:50.282 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2021-01-15T19:04:50.297 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2021-01-15T19:04:50.357 INFO:teuthology.orchestra.run.smithi136.stderr:bash: ntpq: command not found 2021-01-15T19:04:50.360 INFO:teuthology.orchestra.run.smithi136.stdout:210 Number of sources = 4 2021-01-15T19:04:50.360 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2021-01-15T19:04:50.360 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2021-01-15T19:04:50.361 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.361 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.361 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.362 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-01-15T19:04:50.363 INFO:teuthology.run_tasks:Running task install... 2021-01-15T19:04:50.373 DEBUG:teuthology.task.install:project ceph 2021-01-15T19:04:50.374 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}} 2021-01-15T19:04:50.375 DEBUG:teuthology.task.install:config {'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'} 2021-01-15T19:04:50.375 INFO:teuthology.task.install:Using flavor: basic 2021-01-15T19:04:50.382 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-01-15T19:04:50.382 INFO:teuthology.task.install:extra packages: [] 2021-01-15T19:04:50.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo lsb_release -is 2021-01-15T19:04:50.419 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS 2021-01-15T19:04:50.420 DEBUG:teuthology.misc:System to be installed: CentOS 2021-01-15T19:04:50.420 DEBUG:teuthology.orchestra.run.smithi136:> sudo lsb_release -is 2021-01-15T19:04:50.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3', 'tag': None, 'wait_for_package': False} 2021-01-15T19:04:50.422 DEBUG:teuthology.orchestra.run.smithi001:> sudo lsb_release -is 2021-01-15T19:04:50.455 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS 2021-01-15T19:04:50.456 DEBUG:teuthology.misc:System to be installed: CentOS 2021-01-15T19:04:50.457 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'basic', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3', 'tag': None, 'wait_for_package': False} 2021-01-15T19:04:50.457 DEBUG:teuthology.orchestra.run.smithi136:> sudo lsb_release -is 2021-01-15T19:04:50.493 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS 2021-01-15T19:04:50.494 DEBUG:teuthology.misc:System to be installed: CentOS 2021-01-15T19:04:50.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=892c51dd3c5f7108e766bea30cd5e0d801b0abd3 2021-01-15T19:04:50.531 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS 2021-01-15T19:04:50.532 DEBUG:teuthology.misc:System to be installed: CentOS 2021-01-15T19:04:50.532 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=892c51dd3c5f7108e766bea30cd5e0d801b0abd3 2021-01-15T19:04:50.663 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/ 2021-01-15T19:04:50.664 INFO:teuthology.task.install.rpm:Package version is 16.0.0-9031.g892c51dd 2021-01-15T19:04:50.689 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/ 2021-01-15T19:04:50.689 INFO:teuthology.task.install.rpm:Package version is 16.0.0-9031.g892c51dd 2021-01-15T19:04:50.801 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2021-01-15T19:04:50.802 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:04:50.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2021-01-15T19:04:50.813 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/master/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2021-01-15T19:04:50.814 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:04:50.815 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2021-01-15T19:04:50.829 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2021-01-15T19:04:50.829 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/;g' /etc/yum.repos.d/ceph.repo ; fi 2021-01-15T19:04:50.840 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2021-01-15T19:04:50.840 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/892c51dd3c5f7108e766bea30cd5e0d801b0abd3/;g' /etc/yum.repos.d/ceph.repo ; fi 2021-01-15T19:04:50.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2021-01-15T19:04:50.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2021-01-15T19:04:50.973 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2021-01-15T19:04:50.979 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2021-01-15T19:04:51.038 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2021-01-15T19:04:51.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2021-01-15T19:04:51.042 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2021-01-15T19:04:51.043 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2021-01-15T19:04:51.388 INFO:teuthology.orchestra.run.smithi136.stdout:107 files removed 2021-01-15T19:04:51.399 INFO:teuthology.orchestra.run.smithi001.stdout:107 files removed 2021-01-15T19:04:51.418 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw 2021-01-15T19:04:51.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw 2021-01-15T19:04:52.478 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 105 kB/s | 73 kB 00:00 2021-01-15T19:04:52.497 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 108 kB/s | 73 kB 00:00 2021-01-15T19:04:52.924 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 37 kB/s | 15 kB 00:00 2021-01-15T19:04:52.927 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 39 kB/s | 15 kB 00:00 2021-01-15T19:04:53.157 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2021-01-15T19:04:53.162 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 6.9 kB/s | 1.4 kB 00:00 2021-01-15T19:04:53.365 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-8 - AppStream 34 MB/s | 6.3 MB 00:00 2021-01-15T19:04:53.376 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS-8 - AppStream 33 MB/s | 6.3 MB 00:00 2021-01-15T19:04:54.472 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-8 - Base 26 MB/s | 2.3 MB 00:00 2021-01-15T19:04:54.524 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS-8 - Base 26 MB/s | 2.3 MB 00:00 2021-01-15T19:04:54.911 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-8 - Extras 132 kB/s | 8.6 kB 00:00 2021-01-15T19:04:54.994 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS-8 - Extras 132 kB/s | 8.6 kB 00:00 2021-01-15T19:04:55.358 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.8 MB 00:00 2021-01-15T19:04:55.450 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 22 MB/s | 8.8 MB 00:00 2021-01-15T19:04:57.213 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 3.9 MB/s | 23 kB 00:00 2021-01-15T19:04:57.421 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 4.4 MB/s | 23 kB 00:00 2021-01-15T19:04:58.325 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:04:58.337 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:04:58.337 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:04:58.337 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:04:58.338 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:04:58.338 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 12 M 2021-01-15T19:04:58.338 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2021-01-15T19:04:58.338 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 3.7 M 2021-01-15T19:04:58.339 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 3.2 M 2021-01-15T19:04:58.339 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:04:58.339 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 6.0 M 2021-01-15T19:04:58.339 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 24 M 2021-01-15T19:04:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 25 k 2021-01-15T19:04:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2021-01-15T19:04:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2021-01-15T19:04:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2021-01-15T19:04:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 793 k 2021-01-15T19:04:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2021-01-15T19:04:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.9.0-2.el8 CentOS-Base 47 k 2021-01-15T19:04:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 486 k 2021-01-15T19:04:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 0.11.4-1.el8 CentOS-AppStream 353 k 2021-01-15T19:04:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 4.0 M 2021-01-15T19:04:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2021-01-15T19:04:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2021-01-15T19:04:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse 2021-01-15T19:04:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 40 k 2021-01-15T19:04:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 77 k 2021-01-15T19:04:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 186 k 2021-01-15T19:04:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2021-01-15T19:04:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 387 k 2021-01-15T19:04:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 368 k 2021-01-15T19:04:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 114 k 2021-01-15T19:04:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:04:58.345 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:04:58.346 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:04:58.346 INFO:teuthology.orchestra.run.smithi136.stdout:Install 22 Packages 2021-01-15T19:04:58.346 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 2 Packages 2021-01-15T19:04:58.346 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:04:58.347 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 57 M 2021-01-15T19:04:58.347 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:04:58.615 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:04:58.628 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:04:58.628 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:04:58.628 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:04:58.629 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:04:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 12 M 2021-01-15T19:04:58.629 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2021-01-15T19:04:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 3.7 M 2021-01-15T19:04:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 3.2 M 2021-01-15T19:04:58.630 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:04:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 6.0 M 2021-01-15T19:04:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 24 M 2021-01-15T19:04:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 25 k 2021-01-15T19:04:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2021-01-15T19:04:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2021-01-15T19:04:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2021-01-15T19:04:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 793 k 2021-01-15T19:04:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2021-01-15T19:04:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.9.0-2.el8 CentOS-Base 47 k 2021-01-15T19:04:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 486 k 2021-01-15T19:04:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 0.11.4-1.el8 CentOS-AppStream 353 k 2021-01-15T19:04:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 4.0 M 2021-01-15T19:04:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2021-01-15T19:04:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2021-01-15T19:04:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse 2021-01-15T19:04:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 40 k 2021-01-15T19:04:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 77 k 2021-01-15T19:04:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 186 k 2021-01-15T19:04:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2021-01-15T19:04:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 387 k 2021-01-15T19:04:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 368 k 2021-01-15T19:04:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 114 k 2021-01-15T19:04:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:04:58.636 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:04:58.636 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:04:58.637 INFO:teuthology.orchestra.run.smithi001.stdout:Install 22 Packages 2021-01-15T19:04:58.637 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 2 Packages 2021-01-15T19:04:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:04:58.638 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 57 M 2021-01-15T19:04:58.638 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:04:59.021 INFO:teuthology.orchestra.run.smithi136.stdout:(1/24): ceph-base-16.0.0-9031.g892c51dd.el8.x86 8.8 MB/s | 6.0 MB 00:00 2021-01-15T19:04:59.069 INFO:teuthology.orchestra.run.smithi136.stdout:(2/24): ceph-selinux-16.0.0-9031.g892c51dd.el8. 575 kB/s | 25 kB 00:00 2021-01-15T19:04:59.174 INFO:teuthology.orchestra.run.smithi136.stdout:(3/24): libcephfs2-16.0.0-9031.g892c51dd.el8.x8 7.1 MB/s | 793 kB 00:00 2021-01-15T19:04:59.234 INFO:teuthology.orchestra.run.smithi136.stdout:(4/24): libradosstriper1-16.0.0-9031.g892c51dd. 8.0 MB/s | 486 kB 00:00 2021-01-15T19:04:59.644 INFO:teuthology.orchestra.run.smithi136.stdout:(5/24): librgw2-16.0.0-9031.g892c51dd.el8.x86_6 9.7 MB/s | 4.0 MB 00:00 2021-01-15T19:04:59.693 INFO:teuthology.orchestra.run.smithi136.stdout:(6/24): python3-ceph-argparse-16.0.0-9031.g892c 834 kB/s | 40 kB 00:00 2021-01-15T19:04:59.743 INFO:teuthology.orchestra.run.smithi136.stdout:(7/24): python3-ceph-common-16.0.0-9031.g892c51 1.5 MB/s | 77 kB 00:00 2021-01-15T19:04:59.798 INFO:teuthology.orchestra.run.smithi136.stdout:(8/24): python3-cephfs-16.0.0-9031.g892c51dd.el 3.3 MB/s | 186 kB 00:00 2021-01-15T19:04:59.861 INFO:teuthology.orchestra.run.smithi136.stdout:(9/24): python3-rados-16.0.0-9031.g892c51dd.el8 6.0 MB/s | 387 kB 00:00 2021-01-15T19:04:59.924 INFO:teuthology.orchestra.run.smithi136.stdout:(10/24): python3-rbd-16.0.0-9031.g892c51dd.el8. 5.7 MB/s | 368 kB 00:00 2021-01-15T19:04:59.978 INFO:teuthology.orchestra.run.smithi136.stdout:(11/24): python3-rgw-16.0.0-9031.g892c51dd.el8. 2.1 MB/s | 114 kB 00:00 2021-01-15T19:05:00.012 INFO:teuthology.orchestra.run.smithi136.stdout:(12/24): librdkafka-0.11.4-1.el8.x86_64.rpm 10 MB/s | 353 kB 00:00 2021-01-15T19:05:00.016 INFO:teuthology.orchestra.run.smithi136.stdout:(13/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 65 MB/s | 259 kB 00:00 2021-01-15T19:05:00.017 INFO:teuthology.orchestra.run.smithi136.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 31 MB/s | 44 kB 00:00 2021-01-15T19:05:00.019 INFO:teuthology.orchestra.run.smithi136.stdout:(15/24): librabbitmq-0.9.0-2.el8.x86_64.rpm 30 MB/s | 47 kB 00:00 2021-01-15T19:05:00.100 INFO:teuthology.orchestra.run.smithi136.stdout:(16/24): fmt-6.2.1-1.el8.x86_64.rpm 1.4 MB/s | 116 kB 00:00 2021-01-15T19:05:00.150 INFO:teuthology.orchestra.run.smithi136.stdout:(17/24): gperftools-libs-2.7-6.el8.x86_64.rpm 6.3 MB/s | 315 kB 00:00 2021-01-15T19:05:00.167 INFO:teuthology.orchestra.run.smithi136.stdout:(18/24): leveldb-1.22-1.el8.x86_64.rpm 10 MB/s | 181 kB 00:00 2021-01-15T19:05:00.201 INFO:teuthology.orchestra.run.smithi136.stdout:(19/24): ceph-radosgw-16.0.0-9031.g892c51dd.el8 6.6 MB/s | 12 MB 00:01 2021-01-15T19:05:00.269 INFO:teuthology.orchestra.run.smithi136.stdout:(20/24): liboath-2.6.2-3.el8.x86_64.rpm 581 kB/s | 59 kB 00:00 2021-01-15T19:05:00.317 INFO:teuthology.orchestra.run.smithi136.stdout:(21/24): libunwind-1.3.1-3.el8.x86_64.rpm 639 kB/s | 75 kB 00:00 2021-01-15T19:05:00.541 INFO:teuthology.orchestra.run.smithi136.stdout:(22/24): ceph-common-16.0.0-9031.g892c51dd.el8. 11 MB/s | 24 MB 00:02 2021-01-15T19:05:00.627 INFO:teuthology.orchestra.run.smithi136.stdout:(23/24): librados2-16.0.0-9031.g892c51dd.el8.x8 10 MB/s | 3.7 MB 00:00 2021-01-15T19:05:00.709 INFO:teuthology.orchestra.run.smithi136.stdout:(24/24): librbd1-16.0.0-9031.g892c51dd.el8.x86_ 8.3 MB/s | 3.2 MB 00:00 2021-01-15T19:05:00.712 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:00.712 INFO:teuthology.orchestra.run.smithi136.stdout:Total 24 MB/s | 57 MB 00:02 2021-01-15T19:05:00.715 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:05:00.759 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:05:00.760 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:05:00.879 INFO:teuthology.orchestra.run.smithi001.stdout:(1/24): ceph-base-16.0.0-9031.g892c51dd.el8.x86 2.7 MB/s | 6.0 MB 00:02 2021-01-15T19:05:00.905 INFO:teuthology.orchestra.run.smithi001.stdout:(2/24): ceph-selinux-16.0.0-9031.g892c51dd.el8. 965 kB/s | 25 kB 00:00 2021-01-15T19:05:00.986 INFO:teuthology.orchestra.run.smithi001.stdout:(3/24): libcephfs2-16.0.0-9031.g892c51dd.el8.x8 9.6 MB/s | 793 kB 00:00 2021-01-15T19:05:01.035 INFO:teuthology.orchestra.run.smithi001.stdout:(4/24): libradosstriper1-16.0.0-9031.g892c51dd. 9.9 MB/s | 486 kB 00:00 2021-01-15T19:05:01.081 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:05:01.084 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:05:01.365 INFO:teuthology.orchestra.run.smithi001.stdout:(5/24): ceph-radosgw-16.0.0-9031.g892c51dd.el8. 4.5 MB/s | 12 MB 00:02 2021-01-15T19:05:01.400 INFO:teuthology.orchestra.run.smithi001.stdout:(6/24): librgw2-16.0.0-9031.g892c51dd.el8.x86_6 11 MB/s | 4.0 MB 00:00 2021-01-15T19:05:01.401 INFO:teuthology.orchestra.run.smithi001.stdout:(7/24): python3-ceph-argparse-16.0.0-9031.g892c 1.1 MB/s | 40 kB 00:00 2021-01-15T19:05:01.425 INFO:teuthology.orchestra.run.smithi001.stdout:(8/24): python3-ceph-common-16.0.0-9031.g892c51 3.1 MB/s | 77 kB 00:00 2021-01-15T19:05:01.427 INFO:teuthology.orchestra.run.smithi001.stdout:(9/24): python3-cephfs-16.0.0-9031.g892c51dd.el 7.0 MB/s | 186 kB 00:00 2021-01-15T19:05:01.452 INFO:teuthology.orchestra.run.smithi001.stdout:(10/24): python3-rados-16.0.0-9031.g892c51dd.el 14 MB/s | 387 kB 00:00 2021-01-15T19:05:01.464 INFO:teuthology.orchestra.run.smithi001.stdout:(11/24): python3-rbd-16.0.0-9031.g892c51dd.el8. 9.7 MB/s | 368 kB 00:00 2021-01-15T19:05:01.477 INFO:teuthology.orchestra.run.smithi001.stdout:(12/24): python3-rgw-16.0.0-9031.g892c51dd.el8. 4.6 MB/s | 114 kB 00:00 2021-01-15T19:05:01.505 INFO:teuthology.orchestra.run.smithi001.stdout:(13/24): librdkafka-0.11.4-1.el8.x86_64.rpm 8.7 MB/s | 353 kB 00:00 2021-01-15T19:05:01.506 INFO:teuthology.orchestra.run.smithi001.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 25 MB/s | 44 kB 00:00 2021-01-15T19:05:01.508 INFO:teuthology.orchestra.run.smithi001.stdout:(15/24): librabbitmq-0.9.0-2.el8.x86_64.rpm 28 MB/s | 47 kB 00:00 2021-01-15T19:05:01.511 INFO:teuthology.orchestra.run.smithi001.stdout:(16/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 7.5 MB/s | 259 kB 00:00 2021-01-15T19:05:01.606 INFO:teuthology.orchestra.run.smithi001.stdout:(17/24): fmt-6.2.1-1.el8.x86_64.rpm 1.2 MB/s | 116 kB 00:00 2021-01-15T19:05:01.628 INFO:teuthology.orchestra.run.smithi001.stdout:(18/24): gperftools-libs-2.7-6.el8.x86_64.rpm 2.6 MB/s | 315 kB 00:00 2021-01-15T19:05:01.646 INFO:teuthology.orchestra.run.smithi001.stdout:(19/24): leveldb-1.22-1.el8.x86_64.rpm 4.5 MB/s | 181 kB 00:00 2021-01-15T19:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout:(20/24): liboath-2.6.2-3.el8.x86_64.rpm 2.9 MB/s | 59 kB 00:00 2021-01-15T19:05:01.666 INFO:teuthology.orchestra.run.smithi001.stdout:(21/24): libunwind-1.3.1-3.el8.x86_64.rpm 3.7 MB/s | 75 kB 00:00 2021-01-15T19:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout:(22/24): librbd1-16.0.0-9031.g892c51dd.el8.x86_ 14 MB/s | 3.2 MB 00:00 2021-01-15T19:05:01.921 INFO:teuthology.orchestra.run.smithi001.stdout:(23/24): librados2-16.0.0-9031.g892c51dd.el8.x8 14 MB/s | 3.7 MB 00:00 2021-01-15T19:05:02.147 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:05:02.293 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2021-01-15T19:05:02.455 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2021-01-15T19:05:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2021-01-15T19:05:03.222 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2021-01-15T19:05:03.362 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 4/26 2021-01-15T19:05:03.459 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 4/26 2021-01-15T19:05:03.668 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 5/26 2021-01-15T19:05:03.786 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 6/26 2021-01-15T19:05:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 6/26 2021-01-15T19:05:04.370 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 7/26 2021-01-15T19:05:04.439 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 7/26 2021-01-15T19:05:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.9.0-2.el8.x86_64 8/26 2021-01-15T19:05:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 9/26 2021-01-15T19:05:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 9/26 2021-01-15T19:05:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 10/26 2021-01-15T19:05:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 10/26 2021-01-15T19:05:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2021-01-15T19:05:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:16.0.0-9031.g892c51dd.el8. 12/26 2021-01-15T19:05:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 13/26 2021-01-15T19:05:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 14/26 2021-01-15T19:05:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 15/26 2021-01-15T19:05:06.090 INFO:teuthology.orchestra.run.smithi001.stdout:(24/24): ceph-common-16.0.0-9031.g892c51dd.el8. 3.2 MB/s | 24 MB 00:07 2021-01-15T19:05:06.094 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:06.095 INFO:teuthology.orchestra.run.smithi001.stdout:Total 7.6 MB/s | 57 MB 00:07 2021-01-15T19:05:06.099 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:05:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_6 16/26 2021-01-15T19:05:06.153 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:05:06.154 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:05:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_6 16/26 2021-01-15T19:05:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2021-01-15T19:05:06.492 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:05:06.495 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:05:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 18/26 2021-01-15T19:05:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2021-01-15T19:05:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:16.0.0-9031.g892c51dd.el8.x8 20/26 2021-01-15T19:05:07.651 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:05:07.803 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2021-01-15T19:05:07.951 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2021-01-15T19:05:08.139 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2021-01-15T19:05:08.565 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2021-01-15T19:05:08.739 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 4/26 2021-01-15T19:05:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 4/26 2021-01-15T19:05:09.067 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 5/26 2021-01-15T19:05:09.205 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 6/26 2021-01-15T19:05:09.607 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 6/26 2021-01-15T19:05:09.838 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 7/26 2021-01-15T19:05:09.926 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 7/26 2021-01-15T19:05:10.073 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.9.0-2.el8.x86_64 8/26 2021-01-15T19:05:10.222 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 9/26 2021-01-15T19:05:10.377 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 21/26 2021-01-15T19:05:10.552 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 21/26 2021-01-15T19:05:10.655 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 9/26 2021-01-15T19:05:10.805 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 10/26 2021-01-15T19:05:10.874 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 10/26 2021-01-15T19:05:11.010 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2021-01-15T19:05:11.149 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:16.0.0-9031.g892c51dd.el8. 12/26 2021-01-15T19:05:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 21/26 2021-01-15T19:05:11.283 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 13/26 2021-01-15T19:05:11.474 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 14/26 2021-01-15T19:05:11.495 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 22/26 2021-01-15T19:05:11.584 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 22/26 2021-01-15T19:05:11.634 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 15/26 2021-01-15T19:05:11.693 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 23/26 2021-01-15T19:05:11.788 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_6 16/26 2021-01-15T19:05:11.843 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_6 16/26 2021-01-15T19:05:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2021-01-15T19:05:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 18/26 2021-01-15T19:05:12.503 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2021-01-15T19:05:12.614 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:16.0.0-9031.g892c51dd.el8.x8 20/26 2021-01-15T19:05:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 21/26 2021-01-15T19:05:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 21/26 2021-01-15T19:05:17.447 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 21/26 2021-01-15T19:05:17.716 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 22/26 2021-01-15T19:05:17.836 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 22/26 2021-01-15T19:05:17.929 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 23/26 2021-01-15T19:05:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 23/26 2021-01-15T19:05:28.255 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2021-01-15T19:05:28.255 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2021-01-15T19:05:28.255 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /dev 2021-01-15T19:05:28.255 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /run 2021-01-15T19:05:28.256 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2021-01-15T19:05:28.256 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2021-01-15T19:05:28.256 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2021-01-15T19:05:28.256 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2021-01-15T19:05:28.256 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /dev 2021-01-15T19:05:28.257 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 24/26 2021-01-15T19:05:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 24/26 2021-01-15T19:05:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2021-01-15T19:05:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2021-01-15T19:05:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2021-01-15T19:05:30.422 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2021-01-15T19:05:30.423 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 1/26 2021-01-15T19:05:30.423 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 2/26 2021-01-15T19:05:30.423 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 3/26 2021-01-15T19:05:30.423 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 4/26 2021-01-15T19:05:30.423 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 5/26 2021-01-15T19:05:30.424 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_6 6/26 2021-01-15T19:05:30.424 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 7/26 2021-01-15T19:05:30.424 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:16.0.0-9031.g892c51dd.el8. 8/26 2021-01-15T19:05:30.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:16.0.0-9031.g892c51dd.el8.x8 9/26 2021-01-15T19:05:30.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 10/26 2021-01-15T19:05:30.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 11/26 2021-01-15T19:05:30.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 12/26 2021-01-15T19:05:30.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 13/26 2021-01-15T19:05:30.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 14/26 2021-01-15T19:05:30.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2021-01-15T19:05:30.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2021-01-15T19:05:30.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.9.0-2.el8.x86_64 17/26 2021-01-15T19:05:30.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2021-01-15T19:05:30.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 19/26 2021-01-15T19:05:30.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2021-01-15T19:05:30.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2021-01-15T19:05:30.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2021-01-15T19:05:30.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 23/26 2021-01-15T19:05:30.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2021-01-15T19:05:30.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 25/26 2021-01-15T19:05:31.122 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2021-01-15T19:05:31.122 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:31.122 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2021-01-15T19:05:31.123 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.123 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:31.123 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:05:31.124 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.124 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.124 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.124 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.124 INFO:teuthology.orchestra.run.smithi136.stdout: fmt-6.2.1-1.el8.x86_64 2021-01-15T19:05:31.125 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.7-6.el8.x86_64 2021-01-15T19:05:31.125 INFO:teuthology.orchestra.run.smithi136.stdout: leveldb-1.22-1.el8.x86_64 2021-01-15T19:05:31.125 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.125 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.2-3.el8.x86_64 2021-01-15T19:05:31.126 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.9.0-2.el8.x86_64 2021-01-15T19:05:31.126 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.126 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-0.11.4-1.el8.x86_64 2021-01-15T19:05:31.126 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.126 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.3.1-3.el8.x86_64 2021-01-15T19:05:31.127 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.8.1-11.el8.x86_64 2021-01-15T19:05:31.127 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.127 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.127 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.127 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-14.el8.noarch 2021-01-15T19:05:31.128 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.128 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.128 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:31.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:31.129 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:05:31.322 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-test 2021-01-15T19:05:31.787 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:34 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:05:32.233 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:05:32.234 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:05:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:05:32.234 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:05:32.234 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:05:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 49 M 2021-01-15T19:05:32.235 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:05:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: jq x86_64 1.5-12.el8 CentOS-AppStream 161 k 2021-01-15T19:05:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: libxslt x86_64 1.1.32-5.el8 CentOS-Base 250 k 2021-01-15T19:05:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2021-01-15T19:05:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.3.3-2.el8 CentOS-AppStream 302 k 2021-01-15T19:05:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2021-01-15T19:05:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:32.236 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:05:32.237 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:05:32.237 INFO:teuthology.orchestra.run.smithi136.stdout:Install 6 Packages 2021-01-15T19:05:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:32.238 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 50 M 2021-01-15T19:05:32.238 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 218 M 2021-01-15T19:05:32.238 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:05:32.271 INFO:teuthology.orchestra.run.smithi136.stdout:(1/6): jq-1.5-12.el8.x86_64.rpm 4.6 MB/s | 161 kB 00:00 2021-01-15T19:05:32.277 INFO:teuthology.orchestra.run.smithi136.stdout:(2/6): socat-1.7.3.3-2.el8.x86_64.rpm 54 MB/s | 302 kB 00:00 2021-01-15T19:05:32.278 INFO:teuthology.orchestra.run.smithi136.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2021-01-15T19:05:32.281 INFO:teuthology.orchestra.run.smithi136.stdout:(4/6): libxslt-1.1.32-5.el8.x86_64.rpm 63 MB/s | 250 kB 00:00 2021-01-15T19:05:32.344 INFO:teuthology.orchestra.run.smithi136.stdout:(5/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 1.0 MB/s | 69 kB 00:00 2021-01-15T19:05:34.524 INFO:teuthology.orchestra.run.smithi136.stdout:(6/6): ceph-test-16.0.0-9031.g892c51dd.el8.x86_ 22 MB/s | 49 MB 00:02 2021-01-15T19:05:34.530 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:34.530 INFO:teuthology.orchestra.run.smithi136.stdout:Total 22 MB/s | 50 MB 00:02 2021-01-15T19:05:34.532 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:05:34.556 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:05:34.556 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:05:34.818 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:05:34.820 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:05:35.206 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 23/26 2021-01-15T19:05:35.206 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2021-01-15T19:05:35.206 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2021-01-15T19:05:35.207 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /dev 2021-01-15T19:05:35.207 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /run 2021-01-15T19:05:35.207 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2021-01-15T19:05:35.207 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2021-01-15T19:05:35.208 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2021-01-15T19:05:35.208 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2021-01-15T19:05:35.208 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /dev 2021-01-15T19:05:35.209 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:35.302 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:05:35.514 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libxslt-1.1.32-5.el8.x86_64 1/6 2021-01-15T19:05:35.673 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2021-01-15T19:05:35.857 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 3/6 2021-01-15T19:05:35.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 4/6 2021-01-15T19:05:36.070 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 4/6 2021-01-15T19:05:36.511 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 24/26 2021-01-15T19:05:36.594 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 24/26 2021-01-15T19:05:36.594 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2021-01-15T19:05:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2021-01-15T19:05:36.804 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2021-01-15T19:05:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2021-01-15T19:05:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 1/26 2021-01-15T19:05:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 2/26 2021-01-15T19:05:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 3/26 2021-01-15T19:05:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 4/26 2021-01-15T19:05:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 5/26 2021-01-15T19:05:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_6 6/26 2021-01-15T19:05:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 7/26 2021-01-15T19:05:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:16.0.0-9031.g892c51dd.el8. 8/26 2021-01-15T19:05:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:16.0.0-9031.g892c51dd.el8.x8 9/26 2021-01-15T19:05:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 10/26 2021-01-15T19:05:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 11/26 2021-01-15T19:05:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 12/26 2021-01-15T19:05:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 13/26 2021-01-15T19:05:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 14/26 2021-01-15T19:05:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2021-01-15T19:05:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2021-01-15T19:05:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.9.0-2.el8.x86_64 17/26 2021-01-15T19:05:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2021-01-15T19:05:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 19/26 2021-01-15T19:05:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2021-01-15T19:05:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2021-01-15T19:05:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2021-01-15T19:05:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 23/26 2021-01-15T19:05:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2021-01-15T19:05:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 25/26 2021-01-15T19:05:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2021-01-15T19:05:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:38.421 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2021-01-15T19:05:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:38.422 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:05:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: fmt-6.2.1-1.el8.x86_64 2021-01-15T19:05:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.7-6.el8.x86_64 2021-01-15T19:05:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: leveldb-1.22-1.el8.x86_64 2021-01-15T19:05:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.2-3.el8.x86_64 2021-01-15T19:05:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.9.0-2.el8.x86_64 2021-01-15T19:05:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-0.11.4-1.el8.x86_64 2021-01-15T19:05:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.3.1-3.el8.x86_64 2021-01-15T19:05:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.8.1-11.el8.x86_64 2021-01-15T19:05:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-14.el8.noarch 2021-01-15T19:05:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:38.427 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:05:38.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-test 2021-01-15T19:05:39.168 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:00:42 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:05:39.644 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:05:39.645 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:05:39.645 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:05:39.646 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:05:39.646 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:05:39.647 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 49 M 2021-01-15T19:05:39.647 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:05:39.648 INFO:teuthology.orchestra.run.smithi001.stdout: jq x86_64 1.5-12.el8 CentOS-AppStream 161 k 2021-01-15T19:05:39.648 INFO:teuthology.orchestra.run.smithi001.stdout: libxslt x86_64 1.1.32-5.el8 CentOS-Base 250 k 2021-01-15T19:05:39.648 INFO:teuthology.orchestra.run.smithi001.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2021-01-15T19:05:39.649 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.3.3-2.el8 CentOS-AppStream 302 k 2021-01-15T19:05:39.649 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2021-01-15T19:05:39.650 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:39.650 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:05:39.650 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:05:39.651 INFO:teuthology.orchestra.run.smithi001.stdout:Install 6 Packages 2021-01-15T19:05:39.652 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:39.654 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 50 M 2021-01-15T19:05:39.654 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 218 M 2021-01-15T19:05:39.655 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:05:39.693 INFO:teuthology.orchestra.run.smithi001.stdout:(1/6): jq-1.5-12.el8.x86_64.rpm 3.5 MB/s | 161 kB 00:00 2021-01-15T19:05:39.695 INFO:teuthology.orchestra.run.smithi001.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 3.8 MB/s | 187 kB 00:00 2021-01-15T19:05:39.699 INFO:teuthology.orchestra.run.smithi001.stdout:(3/6): socat-1.7.3.3-2.el8.x86_64.rpm 49 MB/s | 302 kB 00:00 2021-01-15T19:05:39.702 INFO:teuthology.orchestra.run.smithi001.stdout:(4/6): libxslt-1.1.32-5.el8.x86_64.rpm 42 MB/s | 250 kB 00:00 2021-01-15T19:05:39.778 INFO:teuthology.orchestra.run.smithi001.stdout:(5/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 879 kB/s | 69 kB 00:00 2021-01-15T19:05:40.715 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jq-1.5-12.el8.x86_64 5/6 2021-01-15T19:05:40.926 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 6/6 2021-01-15T19:05:41.164 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 6/6 2021-01-15T19:05:41.165 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 1/6 2021-01-15T19:05:41.165 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jq-1.5-12.el8.x86_64 2/6 2021-01-15T19:05:41.166 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/6 2021-01-15T19:05:41.166 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 4/6 2021-01-15T19:05:41.167 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libxslt-1.1.32-5.el8.x86_64 5/6 2021-01-15T19:05:41.489 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 6/6 2021-01-15T19:05:41.489 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:41.490 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:05:41.491 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 jq-1.5-12.el8.x86_64 2021-01-15T19:05:41.491 INFO:teuthology.orchestra.run.smithi136.stdout: libxslt-1.1.32-5.el8.x86_64 oniguruma-6.8.2-2.el8.x86_64 2021-01-15T19:05:41.491 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.3.3-2.el8.x86_64 xmlstarlet-1.6.1-11.el8.x86_64 2021-01-15T19:05:41.492 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:41.492 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:05:41.659 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph 2021-01-15T19:05:41.977 INFO:teuthology.orchestra.run.smithi001.stdout:(6/6): ceph-test-16.0.0-9031.g892c51dd.el8.x86_ 21 MB/s | 49 MB 00:02 2021-01-15T19:05:41.981 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:41.981 INFO:teuthology.orchestra.run.smithi001.stdout:Total 22 MB/s | 50 MB 00:02 2021-01-15T19:05:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:05:42.012 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:05:42.013 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:05:42.110 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:45 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:05:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:05:42.294 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:05:42.589 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:05:42.593 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2021-01-15T19:05:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:05:42.594 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2021-01-15T19:05:42.594 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:05:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 6.4 k 2021-01-15T19:05:42.595 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:05:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 2.4 M 2021-01-15T19:05:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 1.5 M 2021-01-15T19:05:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 200 k 2021-01-15T19:05:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 4.9 M 2021-01-15T19:05:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 18 M 2021-01-15T19:05:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2021-01-15T19:05:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2021-01-15T19:05:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-chardet noarch 3.0.4-7.el8 CentOS-Base 195 k 2021-01-15T19:05:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.5.1-1.el8 epel 171 k 2021-01-15T19:05:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2021-01-15T19:05:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2021-01-15T19:05:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2021-01-15T19:05:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2021-01-15T19:05:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: python3-lxml x86_64 4.2.3-1.el8 CentOS-AppStream 1.5 M 2021-01-15T19:05:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2021-01-15T19:05:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2021-01-15T19:05:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2021-01-15T19:05:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2021-01-15T19:05:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2021-01-15T19:05:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pysocks noarch 1.6.8-3.el8 CentOS-Base 34 k 2021-01-15T19:05:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2021-01-15T19:05:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.20.0-2.1.el8_1 CentOS-Base 123 k 2021-01-15T19:05:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2021-01-15T19:05:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2021-01-15T19:05:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2021-01-15T19:05:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2021-01-15T19:05:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.24.2-4.el8 CentOS-Base 176 k 2021-01-15T19:05:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2021-01-15T19:05:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webencodings noarch 0.5.1-6.el8 CentOS-AppStream 27 k 2021-01-15T19:05:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2021-01-15T19:05:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2021-01-15T19:05:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2021-01-15T19:05:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2021-01-15T19:05:42.604 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2021-01-15T19:05:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2021-01-15T19:05:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-html5lib noarch 1:0.999999999-6.el8 CentOS-AppStream 214 k 2021-01-15T19:05:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:42.605 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:05:42.605 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2021-01-15T19:05:42.605 INFO:teuthology.orchestra.run.smithi136.stdout:Install 36 Packages 2021-01-15T19:05:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:42.606 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 32 M 2021-01-15T19:05:42.607 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 113 M 2021-01-15T19:05:42.607 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:05:42.701 INFO:teuthology.orchestra.run.smithi136.stdout:(1/36): ceph-16.0.0-9031.g892c51dd.el8.x86_64.r 63 kB/s | 6.4 kB 00:00 2021-01-15T19:05:42.788 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:05:42.911 INFO:teuthology.orchestra.run.smithi136.stdout:(2/36): ceph-mgr-16.0.0-9031.g892c51dd.el8.x86_ 4.9 MB/s | 1.5 MB 00:00 2021-01-15T19:05:42.960 INFO:teuthology.orchestra.run.smithi136.stdout:(3/36): ceph-mds-16.0.0-9031.g892c51dd.el8.x86_ 6.6 MB/s | 2.4 MB 00:00 2021-01-15T19:05:43.021 INFO:teuthology.orchestra.run.smithi136.stdout:(4/36): ceph-mgr-modules-core-16.0.0-9031.g892c 3.2 MB/s | 200 kB 00:00 2021-01-15T19:05:43.062 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libxslt-1.1.32-5.el8.x86_64 1/6 2021-01-15T19:05:43.064 INFO:teuthology.orchestra.run.smithi136.stdout:(5/36): python3-html5lib-0.999999999-6.el8.noar 4.8 MB/s | 214 kB 00:00 2021-01-15T19:05:43.085 INFO:teuthology.orchestra.run.smithi136.stdout:(6/36): python3-lxml-4.2.3-1.el8.x86_64.rpm 71 MB/s | 1.5 MB 00:00 2021-01-15T19:05:43.088 INFO:teuthology.orchestra.run.smithi136.stdout:(7/36): python3-mako-1.0.6-13.el8.noarch.rpm 57 MB/s | 157 kB 00:00 2021-01-15T19:05:43.089 INFO:teuthology.orchestra.run.smithi136.stdout:(8/36): python3-markupsafe-0.23-19.el8.x86_64.r 35 MB/s | 39 kB 00:00 2021-01-15T19:05:43.093 INFO:teuthology.orchestra.run.smithi136.stdout:(9/36): python3-pytz-2017.2-9.el8.noarch.rpm 21 MB/s | 54 kB 00:00 2021-01-15T19:05:43.095 INFO:teuthology.orchestra.run.smithi136.stdout:(10/36): python3-webencodings-0.5.1-6.el8.noarc 12 MB/s | 27 kB 00:00 2021-01-15T19:05:43.110 INFO:teuthology.orchestra.run.smithi136.stdout:(11/36): python3-werkzeug-0.12.2-4.el8.noarch.r 29 MB/s | 457 kB 00:00 2021-01-15T19:05:43.122 INFO:teuthology.orchestra.run.smithi136.stdout:(12/36): python3-chardet-3.0.4-7.el8.noarch.rpm 16 MB/s | 195 kB 00:00 2021-01-15T19:05:43.133 INFO:teuthology.orchestra.run.smithi136.stdout:(13/36): python3-pysocks-1.6.8-3.el8.noarch.rpm 3.2 MB/s | 34 kB 00:00 2021-01-15T19:05:43.142 INFO:teuthology.orchestra.run.smithi136.stdout:(14/36): python3-requests-2.20.0-2.1.el8_1.noar 14 MB/s | 123 kB 00:00 2021-01-15T19:05:43.145 INFO:teuthology.orchestra.run.smithi136.stdout:(15/36): python3-urllib3-1.24.2-4.el8.noarch.rp 61 MB/s | 176 kB 00:00 2021-01-15T19:05:43.212 INFO:teuthology.orchestra.run.smithi136.stdout:(16/36): ceph-mon-16.0.0-9031.g892c51dd.el8.x86 9.7 MB/s | 4.9 MB 00:00 2021-01-15T19:05:43.222 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2021-01-15T19:05:43.224 INFO:teuthology.orchestra.run.smithi136.stdout:(17/36): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 556 kB/s | 44 kB 00:00 2021-01-15T19:05:43.282 INFO:teuthology.orchestra.run.smithi136.stdout:(18/36): python3-cheroot-8.5.1-1.el8.noarch.rpm 2.9 MB/s | 171 kB 00:00 2021-01-15T19:05:43.294 INFO:teuthology.orchestra.run.smithi136.stdout:(19/36): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.2 MB/s | 185 kB 00:00 2021-01-15T19:05:43.310 INFO:teuthology.orchestra.run.smithi136.stdout:(20/36): python3-cssselect-0.9.2-10.el8.noarch. 2.4 MB/s | 40 kB 00:00 2021-01-15T19:05:43.323 INFO:teuthology.orchestra.run.smithi136.stdout:(21/36): python3-cherrypy-18.4.0-1.el8.noarch.r 9.3 MB/s | 384 kB 00:00 2021-01-15T19:05:43.327 INFO:teuthology.orchestra.run.smithi136.stdout:(22/36): python3-jaraco-6.2-6.el8.noarch.rpm 686 kB/s | 11 kB 00:00 2021-01-15T19:05:43.343 INFO:teuthology.orchestra.run.smithi136.stdout:(23/36): python3-jaraco-functools-2.0-4.el8.noa 885 kB/s | 18 kB 00:00 2021-01-15T19:05:43.344 INFO:teuthology.orchestra.run.smithi136.stdout:(24/36): python3-logutils-0.3.5-11.el8.noarch.r 2.8 MB/s | 49 kB 00:00 2021-01-15T19:05:43.363 INFO:teuthology.orchestra.run.smithi136.stdout:(25/36): python3-more-itertools-7.2.0-3.el8.noa 2.9 MB/s | 59 kB 00:00 2021-01-15T19:05:43.377 INFO:teuthology.orchestra.run.smithi136.stdout:(26/36): python3-pecan-1.3.2-9.el8.noarch.rpm 8.4 MB/s | 283 kB 00:00 2021-01-15T19:05:43.383 INFO:teuthology.orchestra.run.smithi136.stdout:(27/36): python3-portend-2.6-1.el8.noarch.rpm 798 kB/s | 16 kB 00:00 2021-01-15T19:05:43.393 INFO:teuthology.orchestra.run.smithi136.stdout:(28/36): python3-simplegeneric-0.8.1-17.el8.noa 1.1 MB/s | 19 kB 00:00 2021-01-15T19:05:43.402 INFO:teuthology.orchestra.run.smithi136.stdout:(29/36): python3-singledispatch-3.4.0.3-18.el8. 1.2 MB/s | 24 kB 00:00 2021-01-15T19:05:43.410 INFO:teuthology.orchestra.run.smithi136.stdout:(30/36): python3-tempora-1.14.1-5.el8.noarch.rp 1.7 MB/s | 29 kB 00:00 2021-01-15T19:05:43.414 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 3/6 2021-01-15T19:05:43.422 INFO:teuthology.orchestra.run.smithi136.stdout:(31/36): python3-trustme-0.6.0-4.el8.noarch.rpm 1.4 MB/s | 27 kB 00:00 2021-01-15T19:05:43.428 INFO:teuthology.orchestra.run.smithi136.stdout:(32/36): python3-waitress-1.2.1-2.el8.1.noarch. 13 MB/s | 241 kB 00:00 2021-01-15T19:05:43.444 INFO:teuthology.orchestra.run.smithi136.stdout:(33/36): python3-webob-1.8.5-1.el8.1.noarch.rpm 11 MB/s | 251 kB 00:00 2021-01-15T19:05:43.447 INFO:teuthology.orchestra.run.smithi136.stdout:(34/36): python3-webtest-2.0.33-1.el8.noarch.rp 4.9 MB/s | 85 kB 00:00 2021-01-15T19:05:43.465 INFO:teuthology.orchestra.run.smithi136.stdout:(35/36): python3-zc-lockfile-2.0-2.el8.noarch.r 1.1 MB/s | 23 kB 00:00 2021-01-15T19:05:43.542 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 4/6 2021-01-15T19:05:43.624 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 4/6 2021-01-15T19:05:43.840 INFO:teuthology.orchestra.run.smithi136.stdout:(36/36): ceph-osd-16.0.0-9031.g892c51dd.el8.x86 19 MB/s | 18 MB 00:00 2021-01-15T19:05:43.843 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:43.844 INFO:teuthology.orchestra.run.smithi136.stdout:Total 26 MB/s | 32 MB 00:01 2021-01-15T19:05:43.853 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:05:43.894 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:05:43.894 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:05:44.144 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:05:44.147 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:05:46.095 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:05:46.308 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/36 2021-01-15T19:05:46.511 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/36 2021-01-15T19:05:46.731 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/36 2021-01-15T19:05:46.922 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/36 2021-01-15T19:05:47.067 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 5/36 2021-01-15T19:05:47.240 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 6/36 2021-01-15T19:05:47.388 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 7/36 2021-01-15T19:05:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 8/36 2021-01-15T19:05:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 9/36 2021-01-15T19:05:47.906 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 10/36 2021-01-15T19:05:48.131 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.5.1-1.el8.noarch 11/36 2021-01-15T19:05:48.280 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 12/36 2021-01-15T19:05:48.435 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 13/36 2021-01-15T19:05:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : jq-1.5-12.el8.x86_64 5/6 2021-01-15T19:05:48.634 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 14/36 2021-01-15T19:05:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 6/6 2021-01-15T19:05:48.914 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.24.2-4.el8.noarch 15/36 2021-01-15T19:05:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 6/6 2021-01-15T19:05:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 1/6 2021-01-15T19:05:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : jq-1.5-12.el8.x86_64 2/6 2021-01-15T19:05:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/6 2021-01-15T19:05:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 4/6 2021-01-15T19:05:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libxslt-1.1.32-5.el8.x86_64 5/6 2021-01-15T19:05:49.092 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 16/36 2021-01-15T19:05:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 17/36 2021-01-15T19:05:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 6/6 2021-01-15T19:05:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:49.461 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:05:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:16.0.0-9031.g892c51dd.el8.x86_64 jq-1.5-12.el8.x86_64 2021-01-15T19:05:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: libxslt-1.1.32-5.el8.x86_64 oniguruma-6.8.2-2.el8.x86_64 2021-01-15T19:05:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.3.3-2.el8.x86_64 xmlstarlet-1.6.1-11.el8.x86_64 2021-01-15T19:05:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:49.462 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:05:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 18/36 2021-01-15T19:05:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 19/36 2021-01-15T19:05:49.756 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph 2021-01-15T19:05:49.905 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 20/36 2021-01-15T19:05:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 21/36 2021-01-15T19:05:50.233 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:00:53 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:05:50.469 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 22/36 2021-01-15T19:05:50.640 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 23/36 2021-01-15T19:05:50.702 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:05:50.706 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2021-01-15T19:05:50.706 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:05:50.707 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2021-01-15T19:05:50.707 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:05:50.707 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 6.4 k 2021-01-15T19:05:50.707 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:05:50.707 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 2.4 M 2021-01-15T19:05:50.708 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 1.5 M 2021-01-15T19:05:50.708 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 200 k 2021-01-15T19:05:50.708 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 4.9 M 2021-01-15T19:05:50.709 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 18 M 2021-01-15T19:05:50.709 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2021-01-15T19:05:50.709 INFO:teuthology.orchestra.run.smithi001.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2021-01-15T19:05:50.709 INFO:teuthology.orchestra.run.smithi001.stdout: python3-chardet noarch 3.0.4-7.el8 CentOS-Base 195 k 2021-01-15T19:05:50.710 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.5.1-1.el8 epel 171 k 2021-01-15T19:05:50.710 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2021-01-15T19:05:50.710 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2021-01-15T19:05:50.711 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2021-01-15T19:05:50.711 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2021-01-15T19:05:50.711 INFO:teuthology.orchestra.run.smithi001.stdout: python3-lxml x86_64 4.2.3-1.el8 CentOS-AppStream 1.5 M 2021-01-15T19:05:50.711 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2021-01-15T19:05:50.711 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2021-01-15T19:05:50.712 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2021-01-15T19:05:50.712 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2021-01-15T19:05:50.712 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2021-01-15T19:05:50.712 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pysocks noarch 1.6.8-3.el8 CentOS-Base 34 k 2021-01-15T19:05:50.713 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2021-01-15T19:05:50.713 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.20.0-2.1.el8_1 CentOS-Base 123 k 2021-01-15T19:05:50.713 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2021-01-15T19:05:50.713 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2021-01-15T19:05:50.714 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2021-01-15T19:05:50.714 INFO:teuthology.orchestra.run.smithi001.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2021-01-15T19:05:50.714 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.24.2-4.el8 CentOS-Base 176 k 2021-01-15T19:05:50.714 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2021-01-15T19:05:50.715 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webencodings noarch 0.5.1-6.el8 CentOS-AppStream 27 k 2021-01-15T19:05:50.715 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2021-01-15T19:05:50.715 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2021-01-15T19:05:50.715 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2021-01-15T19:05:50.716 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2021-01-15T19:05:50.716 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2021-01-15T19:05:50.716 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2021-01-15T19:05:50.716 INFO:teuthology.orchestra.run.smithi001.stdout: python3-html5lib noarch 1:0.999999999-6.el8 CentOS-AppStream 214 k 2021-01-15T19:05:50.717 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:50.717 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:05:50.717 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2021-01-15T19:05:50.717 INFO:teuthology.orchestra.run.smithi001.stdout:Install 36 Packages 2021-01-15T19:05:50.718 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:05:50.718 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 32 M 2021-01-15T19:05:50.719 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 113 M 2021-01-15T19:05:50.719 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:05:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/36 2021-01-15T19:05:50.814 INFO:teuthology.orchestra.run.smithi001.stdout:(1/36): ceph-16.0.0-9031.g892c51dd.el8.x86_64.r 64 kB/s | 6.4 kB 00:00 2021-01-15T19:05:50.948 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 25/36 2021-01-15T19:05:51.137 INFO:teuthology.orchestra.run.smithi001.stdout:(2/36): ceph-mgr-16.0.0-9031.g892c51dd.el8.x86_ 3.6 MB/s | 1.5 MB 00:00 2021-01-15T19:05:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-2.6-1.el8.noarch 26/36 2021-01-15T19:05:51.236 INFO:teuthology.orchestra.run.smithi001.stdout:(3/36): ceph-mds-16.0.0-9031.g892c51dd.el8.x86_ 4.6 MB/s | 2.4 MB 00:00 2021-01-15T19:05:51.265 INFO:teuthology.orchestra.run.smithi001.stdout:(4/36): ceph-mgr-modules-core-16.0.0-9031.g892c 6.5 MB/s | 200 kB 00:00 2021-01-15T19:05:51.306 INFO:teuthology.orchestra.run.smithi001.stdout:(5/36): ceph-mon-16.0.0-9031.g892c51dd.el8.x86_ 10 MB/s | 4.9 MB 00:00 2021-01-15T19:05:51.321 INFO:teuthology.orchestra.run.smithi001.stdout:(6/36): python3-html5lib-0.999999999-6.el8.noar 3.7 MB/s | 214 kB 00:00 2021-01-15T19:05:51.325 INFO:teuthology.orchestra.run.smithi001.stdout:(7/36): python3-mako-1.0.6-13.el8.noarch.rpm 46 MB/s | 157 kB 00:00 2021-01-15T19:05:51.326 INFO:teuthology.orchestra.run.smithi001.stdout:(8/36): python3-markupsafe-0.23-19.el8.x86_64.r 31 MB/s | 39 kB 00:00 2021-01-15T19:05:51.329 INFO:teuthology.orchestra.run.smithi001.stdout:(9/36): python3-pytz-2017.2-9.el8.noarch.rpm 22 MB/s | 54 kB 00:00 2021-01-15T19:05:51.330 INFO:teuthology.orchestra.run.smithi001.stdout:(10/36): python3-webencodings-0.5.1-6.el8.noarc 17 MB/s | 27 kB 00:00 2021-01-15T19:05:51.341 INFO:teuthology.orchestra.run.smithi001.stdout:(11/36): python3-werkzeug-0.12.2-4.el8.noarch.r 45 MB/s | 457 kB 00:00 2021-01-15T19:05:51.348 INFO:teuthology.orchestra.run.smithi001.stdout:(12/36): python3-chardet-3.0.4-7.el8.noarch.rpm 25 MB/s | 195 kB 00:00 2021-01-15T19:05:51.349 INFO:teuthology.orchestra.run.smithi001.stdout:(13/36): python3-pysocks-1.6.8-3.el8.noarch.rpm 27 MB/s | 34 kB 00:00 2021-01-15T19:05:51.353 INFO:teuthology.orchestra.run.smithi001.stdout:(14/36): python3-requests-2.20.0-2.1.el8_1.noar 44 MB/s | 123 kB 00:00 2021-01-15T19:05:51.363 INFO:teuthology.orchestra.run.smithi001.stdout:(15/36): python3-urllib3-1.24.2-4.el8.noarch.rp 16 MB/s | 176 kB 00:00 2021-01-15T19:05:51.388 INFO:teuthology.orchestra.run.smithi001.stdout:(16/36): python3-lxml-4.2.3-1.el8.x86_64.rpm 19 MB/s | 1.5 MB 00:00 2021-01-15T19:05:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 27/36 2021-01-15T19:05:51.429 INFO:teuthology.orchestra.run.smithi001.stdout:(17/36): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 668 kB/s | 44 kB 00:00 2021-01-15T19:05:51.469 INFO:teuthology.orchestra.run.smithi001.stdout:(18/36): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.2 MB/s | 185 kB 00:00 2021-01-15T19:05:51.477 INFO:teuthology.orchestra.run.smithi001.stdout:(19/36): python3-cheroot-8.5.1-1.el8.noarch.rpm 3.5 MB/s | 171 kB 00:00 2021-01-15T19:05:51.494 INFO:teuthology.orchestra.run.smithi001.stdout:(20/36): python3-cssselect-0.9.2-10.el8.noarch. 2.3 MB/s | 40 kB 00:00 2021-01-15T19:05:51.503 INFO:teuthology.orchestra.run.smithi001.stdout:(21/36): python3-cherrypy-18.4.0-1.el8.noarch.r 11 MB/s | 384 kB 00:00 2021-01-15T19:05:51.511 INFO:teuthology.orchestra.run.smithi001.stdout:(22/36): python3-jaraco-6.2-6.el8.noarch.rpm 678 kB/s | 11 kB 00:00 2021-01-15T19:05:51.519 INFO:teuthology.orchestra.run.smithi001.stdout:(23/36): python3-jaraco-functools-2.0-4.el8.noa 1.1 MB/s | 18 kB 00:00 2021-01-15T19:05:51.528 INFO:teuthology.orchestra.run.smithi001.stdout:(24/36): python3-logutils-0.3.5-11.el8.noarch.r 2.7 MB/s | 49 kB 00:00 2021-01-15T19:05:51.536 INFO:teuthology.orchestra.run.smithi001.stdout:(25/36): python3-more-itertools-7.2.0-3.el8.noa 3.4 MB/s | 59 kB 00:00 2021-01-15T19:05:51.553 INFO:teuthology.orchestra.run.smithi001.stdout:(26/36): python3-portend-2.6-1.el8.noarch.rpm 940 kB/s | 16 kB 00:00 2021-01-15T19:05:51.562 INFO:teuthology.orchestra.run.smithi001.stdout:(27/36): python3-pecan-1.3.2-9.el8.noarch.rpm 8.3 MB/s | 283 kB 00:00 2021-01-15T19:05:51.570 INFO:teuthology.orchestra.run.smithi001.stdout:(28/36): python3-simplegeneric-0.8.1-17.el8.noa 1.1 MB/s | 19 kB 00:00 2021-01-15T19:05:51.579 INFO:teuthology.orchestra.run.smithi001.stdout:(29/36): python3-singledispatch-3.4.0.3-18.el8. 1.4 MB/s | 24 kB 00:00 2021-01-15T19:05:51.587 INFO:teuthology.orchestra.run.smithi001.stdout:(30/36): python3-tempora-1.14.1-5.el8.noarch.rp 1.7 MB/s | 29 kB 00:00 2021-01-15T19:05:51.595 INFO:teuthology.orchestra.run.smithi001.stdout:(31/36): python3-trustme-0.6.0-4.el8.noarch.rpm 1.6 MB/s | 27 kB 00:00 2021-01-15T19:05:51.605 INFO:teuthology.orchestra.run.smithi001.stdout:(32/36): python3-waitress-1.2.1-2.el8.1.noarch. 13 MB/s | 241 kB 00:00 2021-01-15T19:05:51.613 INFO:teuthology.orchestra.run.smithi001.stdout:(33/36): python3-webob-1.8.5-1.el8.1.noarch.rpm 14 MB/s | 251 kB 00:00 2021-01-15T19:05:51.622 INFO:teuthology.orchestra.run.smithi001.stdout:(34/36): python3-webtest-2.0.33-1.el8.noarch.rp 4.9 MB/s | 85 kB 00:00 2021-01-15T19:05:51.630 INFO:teuthology.orchestra.run.smithi001.stdout:(35/36): python3-zc-lockfile-2.0-2.el8.noarch.r 1.4 MB/s | 23 kB 00:00 2021-01-15T19:05:51.648 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/36 2021-01-15T19:05:51.996 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/36 2021-01-15T19:05:52.156 INFO:teuthology.orchestra.run.smithi001.stdout:(36/36): ceph-osd-16.0.0-9031.g892c51dd.el8.x86 18 MB/s | 18 MB 00:01 2021-01-15T19:05:52.160 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:52.160 INFO:teuthology.orchestra.run.smithi001.stdout:Total 22 MB/s | 32 MB 00:01 2021-01-15T19:05:52.170 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:05:52.214 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:05:52.215 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:05:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 30/36 2021-01-15T19:05:52.487 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:05:52.489 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:05:52.648 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-9031.g892c51dd.el8. 31/36 2021-01-15T19:05:52.811 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 32/36 2021-01-15T19:05:54.103 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:05:54.331 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 32/36 2021-01-15T19:05:54.494 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/36 2021-01-15T19:05:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 33/36 2021-01-15T19:05:54.649 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/36 2021-01-15T19:05:54.877 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/36 2021-01-15T19:05:54.994 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 33/36 2021-01-15T19:05:55.070 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/36 2021-01-15T19:05:55.199 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 34/36 2021-01-15T19:05:55.207 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 5/36 2021-01-15T19:05:55.371 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 6/36 2021-01-15T19:05:55.417 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 34/36 2021-01-15T19:05:55.521 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 7/36 2021-01-15T19:05:55.592 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 35/36 2021-01-15T19:05:55.612 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 35/36 2021-01-15T19:05:55.736 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 8/36 2021-01-15T19:05:55.766 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 36/36 2021-01-15T19:05:55.864 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 9/36 2021-01-15T19:05:56.075 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 10/36 2021-01-15T19:05:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 36/36 2021-01-15T19:05:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 1/36 2021-01-15T19:05:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 2/36 2021-01-15T19:05:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 3/36 2021-01-15T19:05:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 4/36 2021-01-15T19:05:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 5/36 2021-01-15T19:05:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-9031.g892c51dd.el8. 6/36 2021-01-15T19:05:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 7/36 2021-01-15T19:05:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 8/36 2021-01-15T19:05:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 9/36 2021-01-15T19:05:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 10/36 2021-01-15T19:05:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 11/36 2021-01-15T19:05:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 12/36 2021-01-15T19:05:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 13/36 2021-01-15T19:05:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 14/36 2021-01-15T19:05:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 15/36 2021-01-15T19:05:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 16/36 2021-01-15T19:05:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.24.2-4.el8.noarch 17/36 2021-01-15T19:05:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/36 2021-01-15T19:05:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/36 2021-01-15T19:05:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.5.1-1.el8.noarch 20/36 2021-01-15T19:05:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 21/36 2021-01-15T19:05:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 22/36 2021-01-15T19:05:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 23/36 2021-01-15T19:05:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 24/36 2021-01-15T19:05:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 25/36 2021-01-15T19:05:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 26/36 2021-01-15T19:05:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 27/36 2021-01-15T19:05:56.131 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-2.6-1.el8.noarch 28/36 2021-01-15T19:05:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 29/36 2021-01-15T19:05:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 30/36 2021-01-15T19:05:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 31/36 2021-01-15T19:05:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 32/36 2021-01-15T19:05:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 33/36 2021-01-15T19:05:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 34/36 2021-01-15T19:05:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 35/36 2021-01-15T19:05:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.5.1-1.el8.noarch 11/36 2021-01-15T19:05:56.453 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 12/36 2021-01-15T19:05:56.641 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 13/36 2021-01-15T19:05:56.868 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 14/36 2021-01-15T19:05:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 36/36 2021-01-15T19:05:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:56.894 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:05:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:05:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:05:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2021-01-15T19:05:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2021-01-15T19:05:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-chardet-3.0.4-7.el8.noarch 2021-01-15T19:05:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.5.1-1.el8.noarch 2021-01-15T19:05:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2021-01-15T19:05:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cssselect-0.9.2-10.el8.noarch 2021-01-15T19:05:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2021-01-15T19:05:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-6.2-6.el8.noarch 2021-01-15T19:05:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2021-01-15T19:05:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-11.el8.noarch 2021-01-15T19:05:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: python3-lxml-4.2.3-1.el8.x86_64 2021-01-15T19:05:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.0.6-13.el8.noarch 2021-01-15T19:05:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-0.23-19.el8.x86_64 2021-01-15T19:05:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2021-01-15T19:05:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.3.2-9.el8.noarch 2021-01-15T19:05:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-2.6-1.el8.noarch 2021-01-15T19:05:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pysocks-1.6.8-3.el8.noarch 2021-01-15T19:05:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2017.2-9.el8.noarch 2021-01-15T19:05:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2021-01-15T19:05:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2021-01-15T19:05:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2021-01-15T19:05:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-1.14.1-5.el8.noarch 2021-01-15T19:05:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: python3-trustme-0.6.0-4.el8.noarch 2021-01-15T19:05:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.24.2-4.el8.noarch 2021-01-15T19:05:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2021-01-15T19:05:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webencodings-0.5.1-6.el8.noarch 2021-01-15T19:05:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.5-1.el8.1.noarch 2021-01-15T19:05:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest-2.0.33-1.el8.noarch 2021-01-15T19:05:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2021-01-15T19:05:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2021-01-15T19:05:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:56.903 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:05:57.106 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-base 2021-01-15T19:05:57.126 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.24.2-4.el8.noarch 15/36 2021-01-15T19:05:57.325 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 16/36 2021-01-15T19:05:57.564 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:00 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:05:57.587 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 17/36 2021-01-15T19:05:57.786 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 18/36 2021-01-15T19:05:57.911 INFO:teuthology.orchestra.run.smithi136.stdout:Package ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:05:57.984 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 19/36 2021-01-15T19:05:58.024 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:05:58.025 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:05:58.026 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:05:58.085 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install cephadm 2021-01-15T19:05:58.325 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 20/36 2021-01-15T19:05:58.536 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:01 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 21/36 2021-01-15T19:05:58.982 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:05:58.984 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:05:58.985 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:05:58.985 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:05:58.986 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:05:58.989 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 66 k 2021-01-15T19:05:58.989 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:58.989 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:05:58.990 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:05:58.990 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:05:58.990 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:05:58.990 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 66 k 2021-01-15T19:05:58.991 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 262 k 2021-01-15T19:05:58.991 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:05:59.089 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 22/36 2021-01-15T19:05:59.131 INFO:teuthology.orchestra.run.smithi136.stdout:cephadm-16.0.0-9031.g892c51dd.el8.noarch.rpm 445 kB/s | 66 kB 00:00 2021-01-15T19:05:59.132 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:05:59.132 INFO:teuthology.orchestra.run.smithi136.stdout:Total 441 kB/s | 66 kB 00:00 2021-01-15T19:05:59.132 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:05:59.140 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:05:59.140 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:05:59.142 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:05:59.144 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:05:59.192 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:05:59.257 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 23/36 2021-01-15T19:05:59.395 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/36 2021-01-15T19:05:59.563 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 25/36 2021-01-15T19:05:59.795 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-2.6-1.el8.noarch 26/36 2021-01-15T19:05:59.996 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 27/36 2021-01-15T19:06:00.242 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/36 2021-01-15T19:06:00.604 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/36 2021-01-15T19:06:00.654 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:00.805 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:00.930 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 30/36 2021-01-15T19:06:00.973 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:01.257 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:01.258 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:01.259 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:01.304 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-9031.g892c51dd.el8. 31/36 2021-01-15T19:06:01.493 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 32/36 2021-01-15T19:06:01.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-immutable-object-cache 2021-01-15T19:06:02.418 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:05 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:02.873 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:02.874 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:02.875 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2021-01-15T19:06:02.875 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:02.875 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:02.876 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 163 k 2021-01-15T19:06:02.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:02.876 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:02.877 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:02.877 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:06:02.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:02.878 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 163 k 2021-01-15T19:06:02.878 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 587 k 2021-01-15T19:06:02.879 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:03.047 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-immutable-object-cache-16.0.0-9031.g892c51 949 kB/s | 163 kB 00:00 2021-01-15T19:06:03.049 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:03.049 INFO:teuthology.orchestra.run.smithi136.stdout:Total 942 kB/s | 163 kB 00:00 2021-01-15T19:06:03.049 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:03.061 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:03.061 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:03.108 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:03.110 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 32/36 2021-01-15T19:06:03.221 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:03.351 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 33/36 2021-01-15T19:06:03.441 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd. 1/1 2021-01-15T19:06:03.803 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd. 1/1 2021-01-15T19:06:03.812 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 33/36 2021-01-15T19:06:03.985 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 34/36 2021-01-15T19:06:04.090 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd. 1/1 2021-01-15T19:06:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:04.091 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:04.092 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:04.223 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 34/36 2021-01-15T19:06:04.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr 2021-01-15T19:06:04.407 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 35/36 2021-01-15T19:06:04.429 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 35/36 2021-01-15T19:06:04.541 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 36/36 2021-01-15T19:06:04.711 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:07 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:04.900 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 36/36 2021-01-15T19:06:04.901 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 1/36 2021-01-15T19:06:04.901 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 2/36 2021-01-15T19:06:04.901 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 3/36 2021-01-15T19:06:04.902 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 4/36 2021-01-15T19:06:04.902 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 5/36 2021-01-15T19:06:04.902 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-9031.g892c51dd.el8. 6/36 2021-01-15T19:06:04.903 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 7/36 2021-01-15T19:06:04.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 8/36 2021-01-15T19:06:04.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 9/36 2021-01-15T19:06:04.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 10/36 2021-01-15T19:06:04.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 11/36 2021-01-15T19:06:04.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 12/36 2021-01-15T19:06:04.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 13/36 2021-01-15T19:06:04.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 14/36 2021-01-15T19:06:04.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 15/36 2021-01-15T19:06:04.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 16/36 2021-01-15T19:06:04.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.24.2-4.el8.noarch 17/36 2021-01-15T19:06:04.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/36 2021-01-15T19:06:04.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/36 2021-01-15T19:06:04.919 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.5.1-1.el8.noarch 20/36 2021-01-15T19:06:04.919 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 21/36 2021-01-15T19:06:04.919 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 22/36 2021-01-15T19:06:04.919 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 23/36 2021-01-15T19:06:04.920 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 24/36 2021-01-15T19:06:04.920 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 25/36 2021-01-15T19:06:04.920 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 26/36 2021-01-15T19:06:04.920 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 27/36 2021-01-15T19:06:04.920 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-2.6-1.el8.noarch 28/36 2021-01-15T19:06:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 29/36 2021-01-15T19:06:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 30/36 2021-01-15T19:06:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 31/36 2021-01-15T19:06:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 32/36 2021-01-15T19:06:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 33/36 2021-01-15T19:06:04.922 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 34/36 2021-01-15T19:06:04.922 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 35/36 2021-01-15T19:06:05.051 INFO:teuthology.orchestra.run.smithi136.stdout:Package ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:05.165 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:05.166 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:05.217 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-dashboard 2021-01-15T19:06:05.673 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 36/36 2021-01-15T19:06:05.674 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:05.674 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:05.674 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:05.674 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:05.675 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:05.675 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:05.675 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:05.675 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:05.676 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2021-01-15T19:06:05.676 INFO:teuthology.orchestra.run.smithi001.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2021-01-15T19:06:05.676 INFO:teuthology.orchestra.run.smithi001.stdout: python3-chardet-3.0.4-7.el8.noarch 2021-01-15T19:06:05.676 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.5.1-1.el8.noarch 2021-01-15T19:06:05.676 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2021-01-15T19:06:05.677 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cssselect-0.9.2-10.el8.noarch 2021-01-15T19:06:05.677 INFO:teuthology.orchestra.run.smithi001.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2021-01-15T19:06:05.677 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-6.2-6.el8.noarch 2021-01-15T19:06:05.677 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2021-01-15T19:06:05.677 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-11.el8.noarch 2021-01-15T19:06:05.678 INFO:teuthology.orchestra.run.smithi001.stdout: python3-lxml-4.2.3-1.el8.x86_64 2021-01-15T19:06:05.683 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.0.6-13.el8.noarch 2021-01-15T19:06:05.683 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-0.23-19.el8.x86_64 2021-01-15T19:06:05.683 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2021-01-15T19:06:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.3.2-9.el8.noarch 2021-01-15T19:06:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-2.6-1.el8.noarch 2021-01-15T19:06:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pysocks-1.6.8-3.el8.noarch 2021-01-15T19:06:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2017.2-9.el8.noarch 2021-01-15T19:06:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2021-01-15T19:06:05.685 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2021-01-15T19:06:05.685 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2021-01-15T19:06:05.685 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-1.14.1-5.el8.noarch 2021-01-15T19:06:05.686 INFO:teuthology.orchestra.run.smithi001.stdout: python3-trustme-0.6.0-4.el8.noarch 2021-01-15T19:06:05.686 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.24.2-4.el8.noarch 2021-01-15T19:06:05.687 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2021-01-15T19:06:05.687 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webencodings-0.5.1-6.el8.noarch 2021-01-15T19:06:05.687 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.5-1.el8.1.noarch 2021-01-15T19:06:05.687 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest-2.0.33-1.el8.noarch 2021-01-15T19:06:05.688 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2021-01-15T19:06:05.688 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2021-01-15T19:06:05.688 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:05.689 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:08 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:05.689 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:05.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-base 2021-01-15T19:06:06.120 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:06.123 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:06:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:06.124 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 2.7 M 2021-01-15T19:06:06.125 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:06:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 24 k 2021-01-15T19:06:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 8.8 k 2021-01-15T19:06:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-Base 43 k 2021-01-15T19:06:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2021-01-15T19:06:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2021-01-15T19:06:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:06.126 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:06.127 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:06.127 INFO:teuthology.orchestra.run.smithi136.stdout:Install 6 Packages 2021-01-15T19:06:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:06.127 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 3.0 M 2021-01-15T19:06:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 14 M 2021-01-15T19:06:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:(1/6): ceph-prometheus-alerts-16.0.0-9031.g892c 81 kB/s | 8.8 kB 00:00 2021-01-15T19:06:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:(2/6): ceph-grafana-dashboards-16.0.0-9031.g892 194 kB/s | 24 kB 00:00 2021-01-15T19:06:06.264 INFO:teuthology.orchestra.run.smithi136.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.4 MB/s | 43 kB 00:00 2021-01-15T19:06:06.307 INFO:teuthology.orchestra.run.smithi136.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 566 kB/s | 34 kB 00:00 2021-01-15T19:06:06.338 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:09 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 2.0 MB/s | 196 kB 00:00 2021-01-15T19:06:06.443 INFO:teuthology.orchestra.run.smithi136.stdout:(6/6): ceph-mgr-dashboard-16.0.0-9031.g892c51dd 8.6 MB/s | 2.7 MB 00:00 2021-01-15T19:06:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:Total 9.4 MB/s | 3.0 MB 00:00 2021-01-15T19:06:06.446 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:06.454 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:06.455 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:06.578 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:06.580 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:06.688 INFO:teuthology.orchestra.run.smithi001.stdout:Package ceph-base-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:06.809 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:06.810 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:06.810 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:06.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install cephadm 2021-01-15T19:06:07.138 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2021-01-15T19:06:07.354 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:10 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:07.855 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:07.855 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:06:07.856 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:07.856 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 66 k 2021-01-15T19:06:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:07.857 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:07.857 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:07.857 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:06:07.857 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:07.858 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 66 k 2021-01-15T19:06:07.858 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 262 k 2021-01-15T19:06:07.858 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:08.005 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm-16.0.0-9031.g892c51dd.el8.noarch.rpm 443 kB/s | 66 kB 00:00 2021-01-15T19:06:08.005 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:08.005 INFO:teuthology.orchestra.run.smithi001.stdout:Total 439 kB/s | 66 kB 00:00 2021-01-15T19:06:08.006 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:08.014 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:08.014 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:08.016 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:08.019 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:08.071 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2021-01-15T19:06:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2021-01-15T19:06:08.859 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-9031.g892c51dd.el8.n 4/6 2021-01-15T19:06:09.793 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:09.881 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-9031.g892c51dd.el8. 5/6 2021-01-15T19:06:09.947 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:10.125 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:10.131 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarc 6/6 2021-01-15T19:06:10.367 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarc 6/6 2021-01-15T19:06:10.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-9031.g892c51dd.el8. 1/6 2021-01-15T19:06:10.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarc 2/6 2021-01-15T19:06:10.367 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-9031.g892c51dd.el8.n 3/6 2021-01-15T19:06:10.368 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2021-01-15T19:06:10.368 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2021-01-15T19:06:10.442 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/1 2021-01-15T19:06:10.443 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:10.443 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:10.443 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:10.443 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:10.443 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:10.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-immutable-object-cache 2021-01-15T19:06:10.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2021-01-15T19:06:10.656 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:10.656 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:10.656 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:10.657 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:10.657 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:10.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt-1.6.1-2.el8.noarch 2021-01-15T19:06:10.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-6.el8.noarch 2021-01-15T19:06:10.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.4.1-12.el8.noarch 2021-01-15T19:06:10.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:10.658 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:10.722 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-diskprediction-local 2021-01-15T19:06:11.109 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:14 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:11.173 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:14 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:11.603 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:11.604 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:11.605 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repo Size 2021-01-15T19:06:11.605 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:11.605 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:11.605 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 163 k 2021-01-15T19:06:11.605 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:11.606 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:11.606 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:11.606 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:06:11.606 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:11.607 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 163 k 2021-01-15T19:06:11.607 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 587 k 2021-01-15T19:06:11.607 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:11.631 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:11.632 INFO:teuthology.orchestra.run.smithi136.stdout:===================================================================================================== 2021-01-15T19:06:11.632 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:06:11.633 INFO:teuthology.orchestra.run.smithi136.stdout:===================================================================================================== 2021-01-15T19:06:11.633 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 8.6 M 2021-01-15T19:06:11.633 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:06:11.633 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 8.3.1-5.1.el8 CentOS-Base 639 k 2021-01-15T19:06:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 8.3.1-5.1.el8 CentOS-Base 169 k 2021-01-15T19:06:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.3-5.el8 CentOS-AppStream 4.3 M 2021-01-15T19:06:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-threads x86_64 0.3.3-5.el8 CentOS-AppStream 4.3 M 2021-01-15T19:06:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.14.3-9.el8 CentOS-AppStream 3.7 M 2021-01-15T19:06:11.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 CentOS-AppStream 225 k 2021-01-15T19:06:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.0.0-20.module_el8.3.0+562+e162826a CentOS-AppStream 15 M 2021-01-15T19:06:11.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:11.635 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:11.635 INFO:teuthology.orchestra.run.smithi136.stdout:===================================================================================================== 2021-01-15T19:06:11.636 INFO:teuthology.orchestra.run.smithi136.stdout:Install 8 Packages 2021-01-15T19:06:11.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:11.636 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 36 M 2021-01-15T19:06:11.637 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 194 M 2021-01-15T19:06:11.637 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:11.746 INFO:teuthology.orchestra.run.smithi136.stdout:(1/8): openblas-0.3.3-5.el8.x86_64.rpm 38 MB/s | 4.3 MB 00:00 2021-01-15T19:06:11.776 INFO:teuthology.orchestra.run.smithi136.stdout:(2/8): openblas-threads-0.3.3-5.el8.x86_64.rpm 31 MB/s | 4.3 MB 00:00 2021-01-15T19:06:11.777 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-immutable-object-cache-16.0.0-9031.g892c51 952 kB/s | 163 kB 00:00 2021-01-15T19:06:11.777 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:11.778 INFO:teuthology.orchestra.run.smithi001.stdout:Total 943 kB/s | 163 kB 00:00 2021-01-15T19:06:11.778 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:11.786 INFO:teuthology.orchestra.run.smithi136.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 23 MB/s | 225 kB 00:00 2021-01-15T19:06:11.792 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:11.793 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:11.820 INFO:teuthology.orchestra.run.smithi136.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 51 MB/s | 3.7 MB 00:00 2021-01-15T19:06:11.833 INFO:teuthology.orchestra.run.smithi136.stdout:(5/8): libgfortran-8.3.1-5.1.el8.x86_64.rpm 49 MB/s | 639 kB 00:00 2021-01-15T19:06:11.838 INFO:teuthology.orchestra.run.smithi136.stdout:(6/8): libquadmath-8.3.1-5.1.el8.x86_64.rpm 32 MB/s | 169 kB 00:00 2021-01-15T19:06:11.843 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:11.846 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:11.957 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:12.063 INFO:teuthology.orchestra.run.smithi136.stdout:(7/8): python3-scipy-1.0.0-20.module_el8.3.0+56 53 MB/s | 15 MB 00:00 2021-01-15T19:06:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd. 1/1 2021-01-15T19:06:12.228 INFO:teuthology.orchestra.run.smithi136.stdout:(8/8): ceph-mgr-diskprediction-local-16.0.0-903 15 MB/s | 8.6 MB 00:00 2021-01-15T19:06:12.229 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:12.230 INFO:teuthology.orchestra.run.smithi136.stdout:Total 61 MB/s | 36 MB 00:00 2021-01-15T19:06:12.239 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:12.257 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:12.258 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd. 1/1 2021-01-15T19:06:12.548 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:12.550 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:12.785 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd. 1/1 2021-01-15T19:06:12.786 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:12.786 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:12.786 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:12.787 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:12.787 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:12.922 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-mgr 2021-01-15T19:06:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:13.221 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-8.3.1-5.1.el8.x86_64 1/8 2021-01-15T19:06:13.338 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libquadmath-8.3.1-5.1.el8.x86_64 1/8 2021-01-15T19:06:13.404 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:16 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:13.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-8.3.1-5.1.el8.x86_64 2/8 2021-01-15T19:06:13.761 INFO:teuthology.orchestra.run.smithi001.stdout:Package ceph-mgr-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:13.880 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:13.881 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:13.881 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:13.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-mgr-dashboard 2021-01-15T19:06:13.949 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libgfortran-8.3.1-5.1.el8.x86_64 2/8 2021-01-15T19:06:14.419 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:17 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:14.729 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.3-5.el8.x86_64 3/8 2021-01-15T19:06:15.072 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:15.073 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:15.073 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:06:15.073 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:15.073 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 2.7 M 2021-01-15T19:06:15.074 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:06:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 24 k 2021-01-15T19:06:15.074 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 8.8 k 2021-01-15T19:06:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-Base 43 k 2021-01-15T19:06:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2021-01-15T19:06:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2021-01-15T19:06:15.075 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:15.076 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:15.076 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:15.076 INFO:teuthology.orchestra.run.smithi001.stdout:Install 6 Packages 2021-01-15T19:06:15.076 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:15.077 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 3.0 M 2021-01-15T19:06:15.077 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 14 M 2021-01-15T19:06:15.078 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:15.187 INFO:teuthology.orchestra.run.smithi001.stdout:(1/6): ceph-prometheus-alerts-16.0.0-9031.g892c 79 kB/s | 8.8 kB 00:00 2021-01-15T19:06:15.201 INFO:teuthology.orchestra.run.smithi001.stdout:(2/6): ceph-grafana-dashboards-16.0.0-9031.g892 192 kB/s | 24 kB 00:00 2021-01-15T19:06:15.225 INFO:teuthology.orchestra.run.smithi001.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2021-01-15T19:06:15.236 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openblas-0.3.3-5.el8.x86_64 3/8 2021-01-15T19:06:15.358 INFO:teuthology.orchestra.run.smithi001.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 215 kB/s | 34 kB 00:00 2021-01-15T19:06:15.369 INFO:teuthology.orchestra.run.smithi001.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.3 MB/s | 196 kB 00:00 2021-01-15T19:06:15.370 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-threads-0.3.3-5.el8.x86_64 4/8 2021-01-15T19:06:15.399 INFO:teuthology.orchestra.run.smithi001.stdout:(6/6): ceph-mgr-dashboard-16.0.0-9031.g892c51dd 8.5 MB/s | 2.7 MB 00:00 2021-01-15T19:06:15.400 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:15.400 INFO:teuthology.orchestra.run.smithi001.stdout:Total 9.3 MB/s | 3.0 MB 00:00 2021-01-15T19:06:15.401 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:15.411 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:15.412 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:15.531 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:15.534 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2021-01-15T19:06:16.083 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openblas-threads-0.3.3-5.el8.x86_64 4/8 2021-01-15T19:06:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2021-01-15T19:06:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2021-01-15T19:06:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2021-01-15T19:06:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-9031.g892c51dd.el8.n 4/6 2021-01-15T19:06:16.571 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-01-15T19:06:17.646 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-9031.g892c51dd.el8. 5/6 2021-01-15T19:06:17.899 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarc 6/6 2021-01-15T19:06:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarc 6/6 2021-01-15T19:06:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-9031.g892c51dd.el8. 1/6 2021-01-15T19:06:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarc 2/6 2021-01-15T19:06:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-9031.g892c51dd.el8.n 3/6 2021-01-15T19:06:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2021-01-15T19:06:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2021-01-15T19:06:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2021-01-15T19:06:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:18.452 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jwt-1.6.1-2.el8.noarch 2021-01-15T19:06:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-6.el8.noarch 2021-01-15T19:06:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.4.1-12.el8.noarch 2021-01-15T19:06:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:18.453 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:18.497 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-01-15T19:06:18.520 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-mgr-diskprediction-local 2021-01-15T19:06:19.002 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:22 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:19.479 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:19.480 INFO:teuthology.orchestra.run.smithi001.stdout:===================================================================================================== 2021-01-15T19:06:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:06:19.481 INFO:teuthology.orchestra.run.smithi001.stdout:===================================================================================================== 2021-01-15T19:06:19.481 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:19.481 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 8.6 M 2021-01-15T19:06:19.482 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:06:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 8.3.1-5.1.el8 CentOS-Base 639 k 2021-01-15T19:06:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 8.3.1-5.1.el8 CentOS-Base 169 k 2021-01-15T19:06:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.3-5.el8 CentOS-AppStream 4.3 M 2021-01-15T19:06:19.482 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-threads x86_64 0.3.3-5.el8 CentOS-AppStream 4.3 M 2021-01-15T19:06:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.14.3-9.el8 CentOS-AppStream 3.7 M 2021-01-15T19:06:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 CentOS-AppStream 225 k 2021-01-15T19:06:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.0.0-20.module_el8.3.0+562+e162826a CentOS-AppStream 15 M 2021-01-15T19:06:19.483 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:19.483 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:19.484 INFO:teuthology.orchestra.run.smithi001.stdout:===================================================================================================== 2021-01-15T19:06:19.484 INFO:teuthology.orchestra.run.smithi001.stdout:Install 8 Packages 2021-01-15T19:06:19.484 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:19.485 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 36 M 2021-01-15T19:06:19.485 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 194 M 2021-01-15T19:06:19.485 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:19.596 INFO:teuthology.orchestra.run.smithi001.stdout:(1/8): openblas-threads-0.3.3-5.el8.x86_64.rpm 38 MB/s | 4.3 MB 00:00 2021-01-15T19:06:19.625 INFO:teuthology.orchestra.run.smithi001.stdout:(2/8): openblas-0.3.3-5.el8.x86_64.rpm 30 MB/s | 4.3 MB 00:00 2021-01-15T19:06:19.633 INFO:teuthology.orchestra.run.smithi001.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 29 MB/s | 225 kB 00:00 2021-01-15T19:06:19.672 INFO:teuthology.orchestra.run.smithi001.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 49 MB/s | 3.7 MB 00:00 2021-01-15T19:06:19.685 INFO:teuthology.orchestra.run.smithi001.stdout:(5/8): libgfortran-8.3.1-5.1.el8.x86_64.rpm 47 MB/s | 639 kB 00:00 2021-01-15T19:06:19.691 INFO:teuthology.orchestra.run.smithi001.stdout:(6/8): libquadmath-8.3.1-5.1.el8.x86_64.rpm 32 MB/s | 169 kB 00:00 2021-01-15T19:06:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.0.0-20.module_el8.3.0+562+e162826a.x 7/8 2021-01-15T19:06:19.903 INFO:teuthology.orchestra.run.smithi001.stdout:(7/8): python3-scipy-1.0.0-20.module_el8.3.0+56 54 MB/s | 15 MB 00:00 2021-01-15T19:06:20.079 INFO:teuthology.orchestra.run.smithi001.stdout:(8/8): ceph-mgr-diskprediction-local-16.0.0-903 14 MB/s | 8.6 MB 00:00 2021-01-15T19:06:20.084 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:20.084 INFO:teuthology.orchestra.run.smithi001.stdout:Total 61 MB/s | 36 MB 00:00 2021-01-15T19:06:20.094 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:20.113 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:20.113 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:20.144 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51d 8/8 2021-01-15T19:06:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51d 8/8 2021-01-15T19:06:20.398 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51d 1/8 2021-01-15T19:06:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.3-5.el8.x86_64 2/8 2021-01-15T19:06:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-threads-0.3.3-5.el8.x86_64 3/8 2021-01-15T19:06:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2021-01-15T19:06:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2021-01-15T19:06:20.399 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.3.0+562+e162826a.x 6/8 2021-01-15T19:06:20.400 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-8.3.1-5.1.el8.x86_64 7/8 2021-01-15T19:06:20.423 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:20.426 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:20.687 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-8.3.1-5.1.el8.x86_64 8/8 2021-01-15T19:06:20.687 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:20.687 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:20.687 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:20.688 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-8.3.1-5.1.el8.x86_64 2021-01-15T19:06:20.688 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-8.3.1-5.1.el8.x86_64 2021-01-15T19:06:20.688 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.3-5.el8.x86_64 2021-01-15T19:06:20.688 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-threads-0.3.3-5.el8.x86_64 2021-01-15T19:06:20.688 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2021-01-15T19:06:20.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2021-01-15T19:06:20.689 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.0.0-20.module_el8.3.0+562+e162826a.x86_64 2021-01-15T19:06:20.689 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:20.689 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:20.757 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-rook 2021-01-15T19:06:21.211 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:24 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:21.650 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:21.652 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================== 2021-01-15T19:06:21.652 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:06:21.653 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================== 2021-01-15T19:06:21.653 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:21.653 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 27 k 2021-01-15T19:06:21.653 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:06:21.654 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2021-01-15T19:06:21.654 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2021-01-15T19:06:21.654 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2021-01-15T19:06:21.655 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2021-01-15T19:06:21.655 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2021-01-15T19:06:21.655 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2021-01-15T19:06:21.655 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-Base 155 k 2021-01-15T19:06:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2021-01-15T19:06:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2021-01-15T19:06:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-Base 43 k 2021-01-15T19:06:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 3.4.2-16.el8 epel 73 k 2021-01-15T19:06:21.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2021-01-15T19:06:21.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:21.657 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:21.657 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================== 2021-01-15T19:06:21.658 INFO:teuthology.orchestra.run.smithi136.stdout:Install 13 Packages 2021-01-15T19:06:21.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:21.658 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 1.9 M 2021-01-15T19:06:21.659 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 24 M 2021-01-15T19:06:21.659 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:21.688 INFO:teuthology.orchestra.run.smithi136.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 819 kB/s | 27 kB 00:00 2021-01-15T19:06:21.692 INFO:teuthology.orchestra.run.smithi136.stdout:(2/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 51 MB/s | 126 kB 00:00 2021-01-15T19:06:21.693 INFO:teuthology.orchestra.run.smithi136.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 563 kB/s | 20 kB 00:00 2021-01-15T19:06:21.693 INFO:teuthology.orchestra.run.smithi136.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 39 MB/s | 110 kB 00:00 2021-01-15T19:06:21.695 INFO:teuthology.orchestra.run.smithi136.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 30 MB/s | 43 kB 00:00 2021-01-15T19:06:21.696 INFO:teuthology.orchestra.run.smithi136.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 32 MB/s | 155 kB 00:00 2021-01-15T19:06:21.745 INFO:teuthology.orchestra.run.smithi136.stdout:(7/13): python3-certifi-2018.10.15-7.el8.noarch 332 kB/s | 16 kB 00:00 2021-01-15T19:06:21.755 INFO:teuthology.orchestra.run.smithi136.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 557 kB/s | 33 kB 00:00 2021-01-15T19:06:21.780 INFO:teuthology.orchestra.run.smithi136.stdout:(9/13): ceph-mgr-rook-16.0.0-9031.g892c51dd.el8 221 kB/s | 27 kB 00:00 2021-01-15T19:06:21.793 INFO:teuthology.orchestra.run.smithi136.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 1.7 MB/s | 81 kB 00:00 2021-01-15T19:06:21.810 INFO:teuthology.orchestra.run.smithi136.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 3.6 MB/s | 61 kB 00:00 2021-01-15T19:06:21.845 INFO:teuthology.orchestra.run.smithi136.stdout:(12/13): python3-rsa-3.4.2-16.el8.noarch.rpm 1.1 MB/s | 73 kB 00:00 2021-01-15T19:06:21.873 INFO:teuthology.orchestra.run.smithi136.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 9.8 MB/s | 1.2 MB 00:00 2021-01-15T19:06:21.875 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:21.876 INFO:teuthology.orchestra.run.smithi136.stdout:Total 8.7 MB/s | 1.9 MB 00:00 2021-01-15T19:06:21.880 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:21.895 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:21.896 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:21.965 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:21.970 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:22.412 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-8.3.1-5.1.el8.x86_64 1/8 2021-01-15T19:06:22.449 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:22.535 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libquadmath-8.3.1-5.1.el8.x86_64 1/8 2021-01-15T19:06:22.665 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2021-01-15T19:06:22.687 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-8.3.1-5.1.el8.x86_64 2/8 2021-01-15T19:06:22.901 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2021-01-15T19:06:23.089 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-3.4.2-16.el8.noarch 3/13 2021-01-15T19:06:23.225 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libgfortran-8.3.1-5.1.el8.x86_64 2/8 2021-01-15T19:06:23.309 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2021-01-15T19:06:23.362 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.3-5.el8.x86_64 3/8 2021-01-15T19:06:23.487 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2021-01-15T19:06:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2021-01-15T19:06:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openblas-0.3.3-5.el8.x86_64 3/8 2021-01-15T19:06:23.966 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2021-01-15T19:06:24.055 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-threads-0.3.3-5.el8.x86_64 4/8 2021-01-15T19:06:24.174 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2021-01-15T19:06:24.854 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openblas-threads-0.3.3-5.el8.x86_64 4/8 2021-01-15T19:06:25.172 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2021-01-15T19:06:25.214 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2021-01-15T19:06:25.395 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-01-15T19:06:25.768 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2021-01-15T19:06:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2021-01-15T19:06:26.753 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2021-01-15T19:06:26.893 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 13/13 2021-01-15T19:06:27.123 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 13/13 2021-01-15T19:06:27.123 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 1/13 2021-01-15T19:06:27.123 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2021-01-15T19:06:27.124 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2021-01-15T19:06:27.124 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2021-01-15T19:06:27.124 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2021-01-15T19:06:27.124 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2021-01-15T19:06:27.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2021-01-15T19:06:27.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2021-01-15T19:06:27.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2021-01-15T19:06:27.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2021-01-15T19:06:27.125 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2021-01-15T19:06:27.126 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-3.4.2-16.el8.noarch 12/13 2021-01-15T19:06:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-01-15T19:06:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2021-01-15T19:06:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:27.464 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-3.1.1-4.el8.noarch 2021-01-15T19:06:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2018.10.15-7.el8.noarch 2021-01-15T19:06:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2021-01-15T19:06:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-2.el8.noarch 2021-01-15T19:06:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-1.10-11.el8.noarch 2021-01-15T19:06:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2021-01-15T19:06:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2021-01-15T19:06:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2021-01-15T19:06:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2021-01-15T19:06:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2021-01-15T19:06:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-3.4.2-16.el8.noarch 2021-01-15T19:06:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2021-01-15T19:06:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:27.468 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:27.552 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-cephadm 2021-01-15T19:06:28.003 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:31 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:28.450 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:28.451 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:06:28.452 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:28.452 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:28.452 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 75 k 2021-01-15T19:06:28.453 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:06:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2021-01-15T19:06:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.5.1-5.el8 CentOS-AppStream 4.8 M 2021-01-15T19:06:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2021-01-15T19:06:28.453 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 CentOS-AppStream 538 k 2021-01-15T19:06:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: python3-remoto noarch 1.1.4-4.el8 epel 48 k 2021-01-15T19:06:28.454 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:28.454 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:28.454 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:28.454 INFO:teuthology.orchestra.run.smithi136.stdout:Install 6 Packages 2021-01-15T19:06:28.455 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:28.455 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 5.8 M 2021-01-15T19:06:28.455 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 24 M 2021-01-15T19:06:28.455 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:28.496 INFO:teuthology.orchestra.run.smithi136.stdout:(1/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 12 MB/s | 538 kB 00:00 2021-01-15T19:06:28.547 INFO:teuthology.orchestra.run.smithi136.stdout:(2/6): python3-babel-2.5.1-5.el8.noarch.rpm 52 MB/s | 4.8 MB 00:00 2021-01-15T19:06:28.547 INFO:teuthology.orchestra.run.smithi136.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 428 kB/s | 21 kB 00:00 2021-01-15T19:06:28.600 INFO:teuthology.orchestra.run.smithi136.stdout:(4/6): ceph-mgr-cephadm-16.0.0-9031.g892c51dd.e 510 kB/s | 75 kB 00:00 2021-01-15T19:06:28.610 INFO:teuthology.orchestra.run.smithi136.stdout:(5/6): python3-remoto-1.1.4-4.el8.noarch.rpm 749 kB/s | 48 kB 00:00 2021-01-15T19:06:28.612 INFO:teuthology.orchestra.run.smithi136.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 5.2 MB/s | 355 kB 00:00 2021-01-15T19:06:28.613 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:28.614 INFO:teuthology.orchestra.run.smithi136.stdout:Total 36 MB/s | 5.8 MB 00:00 2021-01-15T19:06:28.617 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:28.626 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:28.627 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:28.703 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:28.705 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.0.0-20.module_el8.3.0+562+e162826a.x 7/8 2021-01-15T19:06:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51d 8/8 2021-01-15T19:06:29.219 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2021-01-15T19:06:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51d 8/8 2021-01-15T19:06:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51d 1/8 2021-01-15T19:06:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.3-5.el8.x86_64 2/8 2021-01-15T19:06:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-threads-0.3.3-5.el8.x86_64 3/8 2021-01-15T19:06:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2021-01-15T19:06:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2021-01-15T19:06:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.3.0+562+e162826a.x 6/8 2021-01-15T19:06:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-8.3.1-5.1.el8.x86_64 7/8 2021-01-15T19:06:29.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2021-01-15T19:06:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-8.3.1-5.1.el8.x86_64 8/8 2021-01-15T19:06:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:29.704 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-8.3.1-5.1.el8.x86_64 2021-01-15T19:06:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-8.3.1-5.1.el8.x86_64 2021-01-15T19:06:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.3-5.el8.x86_64 2021-01-15T19:06:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-threads-0.3.3-5.el8.x86_64 2021-01-15T19:06:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2021-01-15T19:06:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2021-01-15T19:06:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.0.0-20.module_el8.3.0+562+e162826a.x86_64 2021-01-15T19:06:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:29.707 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:29.773 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-mgr-rook 2021-01-15T19:06:30.254 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:33 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:30.399 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 3/6 2021-01-15T19:06:30.726 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 4/6 2021-01-15T19:06:30.808 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================== 2021-01-15T19:06:30.809 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:06:30.809 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================== 2021-01-15T19:06:30.809 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:30.810 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 27 k 2021-01-15T19:06:30.810 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:06:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2021-01-15T19:06:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2021-01-15T19:06:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2021-01-15T19:06:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2021-01-15T19:06:30.811 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2021-01-15T19:06:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2021-01-15T19:06:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-Base 155 k 2021-01-15T19:06:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2021-01-15T19:06:30.812 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2021-01-15T19:06:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-Base 43 k 2021-01-15T19:06:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 3.4.2-16.el8 epel 73 k 2021-01-15T19:06:30.813 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2021-01-15T19:06:30.814 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:30.814 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:30.814 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================== 2021-01-15T19:06:30.815 INFO:teuthology.orchestra.run.smithi001.stdout:Install 13 Packages 2021-01-15T19:06:30.815 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:30.815 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 1.9 M 2021-01-15T19:06:30.816 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 24 M 2021-01-15T19:06:30.816 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:30.816 INFO:teuthology.orchestra.run.smithi001.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 774 kB/s | 27 kB 00:00 2021-01-15T19:06:30.816 INFO:teuthology.orchestra.run.smithi001.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 544 kB/s | 20 kB 00:00 2021-01-15T19:06:30.816 INFO:teuthology.orchestra.run.smithi001.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 36 MB/s | 126 kB 00:00 2021-01-15T19:06:30.817 INFO:teuthology.orchestra.run.smithi001.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 40 MB/s | 110 kB 00:00 2021-01-15T19:06:30.817 INFO:teuthology.orchestra.run.smithi001.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 25 MB/s | 43 kB 00:00 2021-01-15T19:06:30.817 INFO:teuthology.orchestra.run.smithi001.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 35 MB/s | 155 kB 00:00 2021-01-15T19:06:30.823 INFO:teuthology.orchestra.run.smithi001.stdout:(7/13): python3-certifi-2018.10.15-7.el8.noarch 335 kB/s | 16 kB 00:00 2021-01-15T19:06:30.833 INFO:teuthology.orchestra.run.smithi001.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 553 kB/s | 33 kB 00:00 2021-01-15T19:06:30.857 INFO:teuthology.orchestra.run.smithi001.stdout:(9/13): ceph-mgr-rook-16.0.0-9031.g892c51dd.el8 219 kB/s | 27 kB 00:00 2021-01-15T19:06:30.871 INFO:teuthology.orchestra.run.smithi001.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 1.7 MB/s | 81 kB 00:00 2021-01-15T19:06:30.888 INFO:teuthology.orchestra.run.smithi001.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 3.5 MB/s | 61 kB 00:00 2021-01-15T19:06:30.922 INFO:teuthology.orchestra.run.smithi001.stdout:(12/13): python3-rsa-3.4.2-16.el8.noarch.rpm 1.1 MB/s | 73 kB 00:00 2021-01-15T19:06:30.952 INFO:teuthology.orchestra.run.smithi001.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 9.8 MB/s | 1.2 MB 00:00 2021-01-15T19:06:30.953 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:30.954 INFO:teuthology.orchestra.run.smithi001.stdout:Total 8.6 MB/s | 1.9 MB 00:00 2021-01-15T19:06:30.960 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:30.963 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2021-01-15T19:06:30.976 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:30.977 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:31.052 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:31.054 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:31.117 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 6/6 2021-01-15T19:06:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 6/6 2021-01-15T19:06:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/6 2021-01-15T19:06:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 2/6 2021-01-15T19:06:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 3/6 2021-01-15T19:06:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2021-01-15T19:06:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2021-01-15T19:06:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2021-01-15T19:06:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 6/6 2021-01-15T19:06:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:31.920 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-apipkg-1.5-6.el8.noarch 2021-01-15T19:06:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.5.1-5.el8.noarch 2021-01-15T19:06:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-execnet-1.7.1-1.el8.noarch 2021-01-15T19:06:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2021-01-15T19:06:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: python3-remoto-1.1.4-4.el8.noarch 2021-01-15T19:06:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:31.922 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:31.995 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-fuse 2021-01-15T19:06:32.454 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:35 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:32.588 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2021-01-15T19:06:32.901 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:32.902 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:32.903 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2021-01-15T19:06:32.903 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:32.903 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:32.903 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 877 k 2021-01-15T19:06:32.904 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2021-01-15T19:06:32.904 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.7-12.el8 CentOS-Base 82 k 2021-01-15T19:06:32.904 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-common x86_64 3.2.1-12.el8 CentOS-Base 21 k 2021-01-15T19:06:32.904 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:32.904 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:32.905 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:32.905 INFO:teuthology.orchestra.run.smithi136.stdout:Install 3 Packages 2021-01-15T19:06:32.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:32.905 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 981 k 2021-01-15T19:06:32.906 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 3.2 M 2021-01-15T19:06:32.906 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:32.937 INFO:teuthology.orchestra.run.smithi136.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 2.5 MB/s | 82 kB 00:00 2021-01-15T19:06:32.938 INFO:teuthology.orchestra.run.smithi136.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 624 kB/s | 21 kB 00:00 2021-01-15T19:06:33.145 INFO:teuthology.orchestra.run.smithi136.stdout:(3/3): ceph-fuse-16.0.0-9031.g892c51dd.el8.x86_ 3.6 MB/s | 877 kB 00:00 2021-01-15T19:06:33.147 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:33.147 INFO:teuthology.orchestra.run.smithi136.stdout:Total 3.9 MB/s | 981 kB 00:00 2021-01-15T19:06:33.149 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:33.170 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:33.171 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:33.244 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:33.246 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:33.404 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-3.4.2-16.el8.noarch 3/13 2021-01-15T19:06:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2021-01-15T19:06:33.414 INFO:teuthology.orchestra.run.smithi136.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2021-01-15T19:06:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:33.654 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2021-01-15T19:06:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2021-01-15T19:06:33.880 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2021-01-15T19:06:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 3/3 2021-01-15T19:06:34.125 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2021-01-15T19:06:34.333 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 3/3 2021-01-15T19:06:34.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/3 2021-01-15T19:06:34.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2021-01-15T19:06:34.404 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2021-01-15T19:06:34.603 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2021-01-15T19:06:34.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2021-01-15T19:06:34.714 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:34.715 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:34.715 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2021-01-15T19:06:34.715 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-common-3.2.1-12.el8.x86_64 2021-01-15T19:06:34.715 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:34.716 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:34.872 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install librados-devel 2021-01-15T19:06:35.336 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:38 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:35.765 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2021-01-15T19:06:35.796 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:35.797 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:35.797 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2021-01-15T19:06:35.797 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:35.797 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:35.798 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 131 k 2021-01-15T19:06:35.798 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:35.798 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:35.798 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:35.799 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:06:35.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:35.799 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 131 k 2021-01-15T19:06:35.799 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 437 k 2021-01-15T19:06:35.799 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:35.970 INFO:teuthology.orchestra.run.smithi136.stdout:librados-devel-16.0.0-9031.g892c51dd.el8.x86_64 775 kB/s | 131 kB 00:00 2021-01-15T19:06:35.971 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:35.971 INFO:teuthology.orchestra.run.smithi136.stdout:Total 769 kB/s | 131 kB 00:00 2021-01-15T19:06:35.971 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:35.976 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:35.976 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:36.026 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:36.028 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:36.043 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2021-01-15T19:06:36.134 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:36.244 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2021-01-15T19:06:36.299 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2021-01-15T19:06:36.534 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:36.594 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 13/13 2021-01-15T19:06:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:36.792 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:36.793 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:36.846 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 13/13 2021-01-15T19:06:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 1/13 2021-01-15T19:06:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2021-01-15T19:06:36.847 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2021-01-15T19:06:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2021-01-15T19:06:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2021-01-15T19:06:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2021-01-15T19:06:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2021-01-15T19:06:36.848 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2021-01-15T19:06:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2021-01-15T19:06:36.849 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2021-01-15T19:06:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2021-01-15T19:06:36.850 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-3.4.2-16.el8.noarch 12/13 2021-01-15T19:06:36.952 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install libcephfs2 2021-01-15T19:06:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2021-01-15T19:06:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:37.208 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-3.1.1-4.el8.noarch 2021-01-15T19:06:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2018.10.15-7.el8.noarch 2021-01-15T19:06:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2021-01-15T19:06:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-2.el8.noarch 2021-01-15T19:06:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-1.10-11.el8.noarch 2021-01-15T19:06:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2021-01-15T19:06:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2021-01-15T19:06:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2021-01-15T19:06:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2021-01-15T19:06:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2021-01-15T19:06:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-3.4.2-16.el8.noarch 2021-01-15T19:06:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2021-01-15T19:06:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:37.212 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:37.274 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-mgr-cephadm 2021-01-15T19:06:37.411 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:40 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:37.757 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:40 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:37.778 INFO:teuthology.orchestra.run.smithi136.stdout:Package libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:37.897 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:37.898 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:37.899 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:37.951 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install libcephfs-devel 2021-01-15T19:06:38.228 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:38.229 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:38.229 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:06:38.229 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:38.229 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:16.0.0-9031.g892c51dd.el8 ceph-noarch 75 k 2021-01-15T19:06:38.230 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:06:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2021-01-15T19:06:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.5.1-5.el8 CentOS-AppStream 4.8 M 2021-01-15T19:06:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2021-01-15T19:06:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 CentOS-AppStream 538 k 2021-01-15T19:06:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: python3-remoto noarch 1.1.4-4.el8 epel 48 k 2021-01-15T19:06:38.232 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:38.232 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:38.232 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:38.233 INFO:teuthology.orchestra.run.smithi001.stdout:Install 6 Packages 2021-01-15T19:06:38.233 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:38.234 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 5.8 M 2021-01-15T19:06:38.234 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 24 M 2021-01-15T19:06:38.234 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:38.404 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:41 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:38.501 INFO:teuthology.orchestra.run.smithi001.stdout:(1/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 1.9 MB/s | 538 kB 00:00 2021-01-15T19:06:38.508 INFO:teuthology.orchestra.run.smithi001.stdout:(2/6): ceph-mgr-cephadm-16.0.0-9031.g892c51dd.e 270 kB/s | 75 kB 00:00 2021-01-15T19:06:38.616 INFO:teuthology.orchestra.run.smithi001.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 185 kB/s | 21 kB 00:00 2021-01-15T19:06:38.710 INFO:teuthology.orchestra.run.smithi001.stdout:(4/6): python3-babel-2.5.1-5.el8.noarch.rpm 10 MB/s | 4.8 MB 00:00 2021-01-15T19:06:38.758 INFO:teuthology.orchestra.run.smithi001.stdout:(5/6): python3-remoto-1.1.4-4.el8.noarch.rpm 337 kB/s | 48 kB 00:00 2021-01-15T19:06:38.760 INFO:teuthology.orchestra.run.smithi001.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.4 MB/s | 355 kB 00:00 2021-01-15T19:06:38.762 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:38.763 INFO:teuthology.orchestra.run.smithi001.stdout:Total 11 MB/s | 5.8 MB 00:00 2021-01-15T19:06:38.766 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:38.777 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:38.777 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:38.861 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:38.863 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:38.865 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:38.866 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2021-01-15T19:06:38.867 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:38.867 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 24 k 2021-01-15T19:06:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:38.868 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:38.868 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:38.868 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:06:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:38.869 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 24 k 2021-01-15T19:06:38.869 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 89 k 2021-01-15T19:06:38.869 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:39.225 INFO:teuthology.orchestra.run.smithi136.stdout:libcephfs-devel-16.0.0-9031.g892c51dd.el8.x86_6 67 kB/s | 24 kB 00:00 2021-01-15T19:06:39.226 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:39.226 INFO:teuthology.orchestra.run.smithi136.stdout:Total 67 kB/s | 24 kB 00:00 2021-01-15T19:06:39.227 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:39.230 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:39.231 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:39.233 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:39.235 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:39.301 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:39.453 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2021-01-15T19:06:39.466 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:39.680 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:39.726 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2021-01-15T19:06:39.915 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:39.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:39.916 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:39.916 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:39.917 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:39.917 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:39.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install librados2 2021-01-15T19:06:40.438 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:43 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 3/6 2021-01-15T19:06:40.768 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:40.884 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:40.885 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:40.885 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:40.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install librbd1 2021-01-15T19:06:41.111 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 4/6 2021-01-15T19:06:41.309 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2021-01-15T19:06:41.393 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:44 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:41.477 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 6/6 2021-01-15T19:06:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 6/6 2021-01-15T19:06:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 1/6 2021-01-15T19:06:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 2/6 2021-01-15T19:06:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 3/6 2021-01-15T19:06:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2021-01-15T19:06:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2021-01-15T19:06:41.736 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:41.852 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:41.853 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:41.853 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:41.908 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-rados 2021-01-15T19:06:42.048 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 6/6 2021-01-15T19:06:42.048 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:42.048 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:42.049 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:16.0.0-9031.g892c51dd.el8.noarch 2021-01-15T19:06:42.049 INFO:teuthology.orchestra.run.smithi001.stdout: python3-apipkg-1.5-6.el8.noarch 2021-01-15T19:06:42.049 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.5.1-5.el8.noarch 2021-01-15T19:06:42.049 INFO:teuthology.orchestra.run.smithi001.stdout: python3-execnet-1.7.1-1.el8.noarch 2021-01-15T19:06:42.049 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2021-01-15T19:06:42.050 INFO:teuthology.orchestra.run.smithi001.stdout: python3-remoto-1.1.4-4.el8.noarch 2021-01-15T19:06:42.050 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:42.050 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:42.109 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-fuse 2021-01-15T19:06:42.362 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:45 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:42.593 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:45 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:42.835 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:42.949 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:42.950 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:42.950 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:43.009 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-rgw 2021-01-15T19:06:43.101 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:43.102 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:43.102 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2021-01-15T19:06:43.103 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:43.103 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:43.103 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 877 k 2021-01-15T19:06:43.103 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2021-01-15T19:06:43.104 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.7-12.el8 CentOS-Base 82 k 2021-01-15T19:06:43.104 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-common x86_64 3.2.1-12.el8 CentOS-Base 21 k 2021-01-15T19:06:43.104 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:43.104 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:43.105 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:43.105 INFO:teuthology.orchestra.run.smithi001.stdout:Install 3 Packages 2021-01-15T19:06:43.105 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:43.106 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 981 k 2021-01-15T19:06:43.106 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 3.2 M 2021-01-15T19:06:43.106 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:43.138 INFO:teuthology.orchestra.run.smithi001.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 2.4 MB/s | 82 kB 00:00 2021-01-15T19:06:43.139 INFO:teuthology.orchestra.run.smithi001.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 606 kB/s | 21 kB 00:00 2021-01-15T19:06:43.345 INFO:teuthology.orchestra.run.smithi001.stdout:(3/3): ceph-fuse-16.0.0-9031.g892c51dd.el8.x86_ 3.6 MB/s | 877 kB 00:00 2021-01-15T19:06:43.347 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:43.347 INFO:teuthology.orchestra.run.smithi001.stdout:Total 3.9 MB/s | 981 kB 00:00 2021-01-15T19:06:43.349 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:43.373 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:43.373 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:43.453 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:43.455 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:43.464 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:46 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:43.667 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:43.668 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2021-01-15T19:06:43.668 INFO:teuthology.orchestra.run.smithi001.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2021-01-15T19:06:43.669 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:43.809 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:43.921 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:43.922 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:43.923 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:43.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-cephfs 2021-01-15T19:06:44.089 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2021-01-15T19:06:44.435 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:47 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:44.568 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 3/3 2021-01-15T19:06:44.784 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:44.900 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:44.900 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:44.901 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:44.934 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 3/3 2021-01-15T19:06:44.934 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/3 2021-01-15T19:06:44.935 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2021-01-15T19:06:44.952 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-rbd 2021-01-15T19:06:45.282 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2021-01-15T19:06:45.283 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:45.283 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:45.284 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2021-01-15T19:06:45.284 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-common-3.2.1-12.el8.x86_64 2021-01-15T19:06:45.284 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:45.285 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:45.406 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:48 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:45.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install librados-devel 2021-01-15T19:06:45.734 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:45.850 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:45.851 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2021-01-15T19:06:45.851 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:45.908 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install rbd-fuse 2021-01-15T19:06:45.936 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:48 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:46.360 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:49 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:46.399 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:46.400 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:46.401 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repo Size 2021-01-15T19:06:46.401 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:46.401 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:46.401 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 131 k 2021-01-15T19:06:46.401 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:46.402 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:46.402 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:46.402 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:06:46.402 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:46.403 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 131 k 2021-01-15T19:06:46.403 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 437 k 2021-01-15T19:06:46.403 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:46.572 INFO:teuthology.orchestra.run.smithi001.stdout:librados-devel-16.0.0-9031.g892c51dd.el8.x86_64 769 kB/s | 131 kB 00:00 2021-01-15T19:06:46.573 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:46.573 INFO:teuthology.orchestra.run.smithi001.stdout:Total 764 kB/s | 131 kB 00:00 2021-01-15T19:06:46.573 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:46.581 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:46.581 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:46.635 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:46.638 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:46.754 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:46.830 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:46.830 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:46.831 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2021-01-15T19:06:46.831 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:46.831 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:46.831 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 89 k 2021-01-15T19:06:46.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:46.832 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:46.832 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:46.832 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:06:46.833 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:46.833 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 89 k 2021-01-15T19:06:46.833 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 298 k 2021-01-15T19:06:46.833 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:46.979 INFO:teuthology.orchestra.run.smithi136.stdout:rbd-fuse-16.0.0-9031.g892c51dd.el8.x86_64.rpm 606 kB/s | 89 kB 00:00 2021-01-15T19:06:46.980 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:46.980 INFO:teuthology.orchestra.run.smithi136.stdout:Total 601 kB/s | 89 kB 00:00 2021-01-15T19:06:46.980 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:46.991 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:46.991 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:47.041 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:47.044 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:47.145 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:47.207 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:47.321 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:47.547 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:47.547 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:47.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install libcephfs2 2021-01-15T19:06:47.823 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:47.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:47.823 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:47.824 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:47.824 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:47.824 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:48.200 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:51 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:48.407 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install rbd-mirror 2021-01-15T19:06:48.557 INFO:teuthology.orchestra.run.smithi001.stdout:Package libcephfs2-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:48.683 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:48.685 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:48.685 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:48.744 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install libcephfs-devel 2021-01-15T19:06:48.869 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:51 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:49.231 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:52 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:49.665 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:49.666 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2021-01-15T19:06:49.666 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:49.667 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 3.6 M 2021-01-15T19:06:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:49.667 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:49.667 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:49.668 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:06:49.668 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:49.668 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 3.6 M 2021-01-15T19:06:49.668 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 18 M 2021-01-15T19:06:49.669 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:49.713 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:49.714 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repo Size 2021-01-15T19:06:49.714 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:49.715 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 24 k 2021-01-15T19:06:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:49.715 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:49.715 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:49.716 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:06:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:49.716 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 24 k 2021-01-15T19:06:49.716 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 89 k 2021-01-15T19:06:49.717 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:49.854 INFO:teuthology.orchestra.run.smithi001.stdout:libcephfs-devel-16.0.0-9031.g892c51dd.el8.x86_6 172 kB/s | 24 kB 00:00 2021-01-15T19:06:49.855 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:49.855 INFO:teuthology.orchestra.run.smithi001.stdout:Total 170 kB/s | 24 kB 00:00 2021-01-15T19:06:49.856 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:49.864 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:49.865 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:49.865 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:49.866 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:50.033 INFO:teuthology.orchestra.run.smithi136.stdout:rbd-mirror-16.0.0-9031.g892c51dd.el8.x86_64.rpm 10 MB/s | 3.6 MB 00:00 2021-01-15T19:06:50.034 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:50.035 INFO:teuthology.orchestra.run.smithi136.stdout:Total 9.9 MB/s | 3.6 MB 00:00 2021-01-15T19:06:50.035 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:50.048 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:50.051 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:50.051 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:50.116 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:50.123 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:50.208 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:50.439 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:50.643 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:50.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:50.679 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:50.680 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:50.680 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:50.680 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:50.680 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:50.744 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install librados2 2021-01-15T19:06:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:51.224 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:54 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:51.456 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:51.456 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:51.573 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:51.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install rbd-nbd 2021-01-15T19:06:51.690 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:51.691 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:51.691 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:51.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install librbd1 2021-01-15T19:06:52.051 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:55 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:52.231 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:55 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:52.494 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2021-01-15T19:06:52.495 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:52.495 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2021-01-15T19:06:52.496 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:52.496 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2021-01-15T19:06:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 174 k 2021-01-15T19:06:52.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:52.497 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2021-01-15T19:06:52.497 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2021-01-15T19:06:52.497 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2021-01-15T19:06:52.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:52.497 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 174 k 2021-01-15T19:06:52.498 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 649 k 2021-01-15T19:06:52.498 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2021-01-15T19:06:52.581 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:52.667 INFO:teuthology.orchestra.run.smithi136.stdout:rbd-nbd-16.0.0-9031.g892c51dd.el8.x86_64.rpm 1.0 MB/s | 174 kB 00:00 2021-01-15T19:06:52.668 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:52.668 INFO:teuthology.orchestra.run.smithi136.stdout:Total 1.0 MB/s | 174 kB 00:00 2021-01-15T19:06:52.669 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2021-01-15T19:06:52.682 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2021-01-15T19:06:52.682 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2021-01-15T19:06:52.698 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:52.699 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:52.699 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:52.732 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2021-01-15T19:06:52.734 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2021-01-15T19:06:52.754 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install python3-rados 2021-01-15T19:06:52.854 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2021-01-15T19:06:53.053 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:53.241 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:56 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:53.290 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:53.550 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2021-01-15T19:06:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:06:53.551 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2021-01-15T19:06:53.614 INFO:teuthology.orchestra.run.smithi001.stdout:Package python3-rados-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:53.701 DEBUG:teuthology.parallel:result is None 2021-01-15T19:06:53.734 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:53.735 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:53.736 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:53.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install python3-rgw 2021-01-15T19:06:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:57 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:Package python3-rgw-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:54.765 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:54.766 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:54.767 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:54.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install python3-cephfs 2021-01-15T19:06:55.313 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:58 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:55.689 INFO:teuthology.orchestra.run.smithi001.stdout:Package python3-cephfs-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:55.807 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:55.808 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:55.809 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:55.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install python3-rbd 2021-01-15T19:06:56.360 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:01:59 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:56.722 INFO:teuthology.orchestra.run.smithi001.stdout:Package python3-rbd-2:16.0.0-9031.g892c51dd.el8.x86_64 is already installed. 2021-01-15T19:06:56.841 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:56.842 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2021-01-15T19:06:56.843 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:56.905 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install rbd-fuse 2021-01-15T19:06:57.389 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:00 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:06:57.866 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:06:57.867 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2021-01-15T19:06:57.867 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:57.868 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:06:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 89 k 2021-01-15T19:06:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:57.868 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:06:57.868 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:06:57.869 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:06:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:57.869 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 89 k 2021-01-15T19:06:57.870 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 298 k 2021-01-15T19:06:57.870 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:06:58.038 INFO:teuthology.orchestra.run.smithi001.stdout:rbd-fuse-16.0.0-9031.g892c51dd.el8.x86_64.rpm 523 kB/s | 89 kB 00:00 2021-01-15T19:06:58.039 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:06:58.039 INFO:teuthology.orchestra.run.smithi001.stdout:Total 519 kB/s | 89 kB 00:00 2021-01-15T19:06:58.039 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:06:58.052 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:06:58.053 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:06:58.106 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:06:58.109 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:06:58.221 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:06:58.432 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:58.988 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:06:58.988 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:58.989 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:06:58.989 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:06:58.989 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:06:58.989 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:06:59.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install rbd-mirror 2021-01-15T19:06:59.684 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:02 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:07:00.173 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:07:00.174 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:07:00.174 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repo Size 2021-01-15T19:07:00.175 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:07:00.175 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:07:00.175 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 3.6 M 2021-01-15T19:07:00.175 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:00.176 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:07:00.176 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:07:00.176 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:07:00.176 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:00.177 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 3.6 M 2021-01-15T19:07:00.177 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 18 M 2021-01-15T19:07:00.177 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:07:00.544 INFO:teuthology.orchestra.run.smithi001.stdout:rbd-mirror-16.0.0-9031.g892c51dd.el8.x86_64.rpm 9.9 MB/s | 3.6 MB 00:00 2021-01-15T19:07:00.544 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:07:00.545 INFO:teuthology.orchestra.run.smithi001.stdout:Total 9.9 MB/s | 3.6 MB 00:00 2021-01-15T19:07:00.545 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:07:00.561 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:07:00.561 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:07:00.630 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:07:00.633 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:07:01.720 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:07:01.950 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:07:02.304 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:07:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:07:02.596 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:02.597 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:07:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:07:02.597 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:02.597 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:07:02.785 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install rbd-nbd 2021-01-15T19:07:03.275 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:06 ago on Fri 15 Jan 2021 07:04:57 PM UTC. 2021-01-15T19:07:03.788 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2021-01-15T19:07:03.789 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:07:03.789 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2021-01-15T19:07:03.789 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:07:03.790 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2021-01-15T19:07:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:16.0.0-9031.g892c51dd.el8 ceph 174 k 2021-01-15T19:07:03.790 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:03.790 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2021-01-15T19:07:03.791 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2021-01-15T19:07:03.791 INFO:teuthology.orchestra.run.smithi001.stdout:Install 1 Package 2021-01-15T19:07:03.791 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:03.791 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 174 k 2021-01-15T19:07:03.791 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 649 k 2021-01-15T19:07:03.792 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2021-01-15T19:07:03.962 INFO:teuthology.orchestra.run.smithi001.stdout:rbd-nbd-16.0.0-9031.g892c51dd.el8.x86_64.rpm 1.0 MB/s | 174 kB 00:00 2021-01-15T19:07:03.963 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2021-01-15T19:07:03.963 INFO:teuthology.orchestra.run.smithi001.stdout:Total 1.0 MB/s | 174 kB 00:00 2021-01-15T19:07:03.963 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2021-01-15T19:07:03.977 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2021-01-15T19:07:03.978 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2021-01-15T19:07:04.030 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2021-01-15T19:07:04.033 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2021-01-15T19:07:04.159 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2021-01-15T19:07:04.364 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:07:04.611 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:07:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 1/1 2021-01-15T19:07:04.921 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:04.922 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2021-01-15T19:07:04.922 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:16.0.0-9031.g892c51dd.el8.x86_64 2021-01-15T19:07:04.923 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:04.923 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2021-01-15T19:07:05.081 DEBUG:teuthology.parallel:result is None 2021-01-15T19:07:05.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo lsb_release -is 2021-01-15T19:07:05.117 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS 2021-01-15T19:07:05.118 DEBUG:teuthology.misc:System to be installed: CentOS 2021-01-15T19:07:05.119 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=892c51dd3c5f7108e766bea30cd5e0d801b0abd3 2021-01-15T19:07:05.292 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2021-01-15T19:07:05.312 INFO:teuthology.orchestra.run.smithi001.stdout:16.0.0-9031.g892c51dd.el8 2021-01-15T19:07:05.313 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-9031.g892c51dd.el8 2021-01-15T19:07:05.314 INFO:teuthology.task.install:The correct ceph version 16.0.0-9031.g892c51dd is installed. 2021-01-15T19:07:05.314 DEBUG:teuthology.orchestra.run.smithi136:> sudo lsb_release -is 2021-01-15T19:07:05.357 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS 2021-01-15T19:07:05.358 DEBUG:teuthology.misc:System to be installed: CentOS 2021-01-15T19:07:05.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=892c51dd3c5f7108e766bea30cd5e0d801b0abd3 2021-01-15T19:07:05.525 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2021-01-15T19:07:05.550 INFO:teuthology.orchestra.run.smithi136.stdout:16.0.0-9031.g892c51dd.el8 2021-01-15T19:07:05.552 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-9031.g892c51dd.el8 2021-01-15T19:07:05.553 INFO:teuthology.task.install:The correct ceph version 16.0.0-9031.g892c51dd is installed. 2021-01-15T19:07:05.554 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-01-15T19:07:05.555 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:05.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-01-15T19:07:05.584 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:05.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-01-15T19:07:05.622 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-01-15T19:07:05.623 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:05.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2021-01-15T19:07:05.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-01-15T19:07:05.724 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:05.725 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2021-01-15T19:07:05.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-01-15T19:07:05.822 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-01-15T19:07:05.823 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:05.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2021-01-15T19:07:05.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-01-15T19:07:05.964 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:05.966 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2021-01-15T19:07:05.994 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-01-15T19:07:06.062 INFO:teuthology.run_tasks:Running task ceph... 2021-01-15T19:07:06.160 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-01-15T19:07:06.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /var/log/ceph 2021-01-15T19:07:06.166 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2021-01-15T19:07:06.193 INFO:tasks.ceph:Disabling ceph logrotate... 2021-01-15T19:07:06.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/logrotate.d/ceph 2021-01-15T19:07:06.243 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2021-01-15T19:07:06.270 INFO:tasks.ceph:Creating extra log directories... 2021-01-15T19:07:06.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-01-15T19:07:06.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-01-15T19:07:06.381 INFO:tasks.ceph:Creating ceph cluster ceph... 2021-01-15T19:07:06.382 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-01-15T19:07:06.382 INFO:tasks.ceph:ctx.config {'arch': 'x86_64', 'archive_path': '/home/teuthworker/archive/teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi/5788674', 'branch': 'master', 'description': 'rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5788674', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2021-01-15_07:01:02-rados-master-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '8.2', 'overrides': {'admin_socket': {'branch': 'master'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'master', 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 80, 'repo': 'git://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'master', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_master/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph.git', 'suite_sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 2, 'm': 1, 'name': 'isaprofile', 'plugin': 'isa', 'technique': 'reed_sol_van'}, 'min_size': 2, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 4000, 'write_append_excl': False}}], 'teuthology_branch': 'master', 'timestamp': '2021-01-15_07:01:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.1447', 'targets': {'smithi001.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj0drSpCVPEDKuZjdqvblKC6kS3T6Cb0x4UKpCtrtU0lPM8wbPCSZ4b6yqoovZZYVp3wKDfi2zpzw/6IZaxnee+yr3KKnQkYMQVx4WcRDncTFIICku+y1wLXZQvojY+GfkNzUeAP0pdXyZQ8lA/zRy79XmO2iWYNmegg7wQAvgYviXIi0yFKxAPJ8ELapU7Jd+qbAiFnCLl7W4kFFAziRcTmahy2Y7zEJfw4YVnr05Oyjkb6WvgIcNWmJpzzzaSTinyyT+/jvI1z1pMfUB0u++i7q4C5ei3wqsQpV7oKJw3huO93WEv766BUwGdGuvf2yvzJeaecsQ0ohI1E4Nl7G+dxnhxQCun3rH+WGMqs3FLB/bXav3VCgJDbj9liqGn5VWeZHfgnq4LrINzBDnDrFikdMMpufnyOKw9eKP2gBLr3J3M9KvneuZVhnBQhp8pa0SWj1DK1Mtz98pcUmH1ErLh6CQ5/rr28CLM3u9cwdlREsgngCbshLC+gyTfs4ViJs=', 'smithi136.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZm+NC4WpSzrSM/d7OPPqqlQcDsC6R38soVwkBKlU5x6fm8Ryk9OGISm6xL0dchgW2FMQeKlwsWODPVfQwOzQD+vZ22anx2ZXB3lVtaIFhxYAHVik65M0IrNJfA6DW7YbW/2J0EOOw9Cm649WmZ14H0fqHo4KX3ltUCClb8dT7ELJKpFdgpsoEiFI2XQxeXETWT7h0Wq18gpGxvyWxnerOWzCJ9sPfqEgw8j7SbbefAk/yrWk20FcMbfNdHQPGKWYW13/xCm95sM1WcoFzDXyjXh5/tfjSEMNzpl0+Oz3Rfrc11DM+tbKtQU+9Yw/v5kRcHil3lw80SRp4VX3Q25WgpuNfLuUOEuS/jAeOMwEUc7ZC5QYzzIBzFcnIjVvxdFKrj+VPPWcEDW1cCuivDt5n+ns7Pt+sNhT8fvhRRziVvbAimsGY71xxjKYYIhR2XUpyXHkIC63XVzrA/cPz794muF7js2igC0WjwjaddHDTWsqaIcumLSXZY1U73stdMK8='}} 2021-01-15T19:07:06.383 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-01-15T19:07:06.386 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-01-15T19:07:06.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/run/ceph 2021-01-15T19:07:06.445 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2021-01-15T19:07:06.472 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:06.473 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2021-01-15T19:07:06.528 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-01-15T19:07:06.528 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-01-15T19:07:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-01-15T19:07:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:06.583 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 112384 Links: 1 2021-01-15T19:07:06.583 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2021-01-15 19:07:02.101358114 +0000 2021-01-15T19:07:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2021-01-15 19:04:29.497822833 +0000 2021-01-15T19:07:06.584 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2021-01-15 19:04:29.497822833 +0000 2021-01-15T19:07:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2021-01-15T19:07:06.594 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2021-01-15T19:07:06.594 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2021-01-15T19:07:06.594 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000110081 s, 4.7 MB/s 2021-01-15T19:07:06.597 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-01-15T19:07:06.653 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-01-15T19:07:06.654 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:06.654 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 112424 Links: 1 2021-01-15T19:07:06.654 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:06.654 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:06.654 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2021-01-15 19:07:02.101358114 +0000 2021-01-15T19:07:06.655 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2021-01-15 19:04:29.838818571 +0000 2021-01-15T19:07:06.655 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2021-01-15 19:04:29.838818571 +0000 2021-01-15T19:07:06.655 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2021-01-15T19:07:06.664 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2021-01-15T19:07:06.664 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2021-01-15T19:07:06.665 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 8.9177e-05 s, 5.7 MB/s 2021-01-15T19:07:06.667 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-01-15T19:07:06.723 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-01-15T19:07:06.724 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:06.724 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 113869 Links: 1 2021-01-15T19:07:06.725 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:06.725 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:06.725 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2021-01-15 19:07:02.101358114 +0000 2021-01-15T19:07:06.726 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2021-01-15 19:04:30.160814546 +0000 2021-01-15T19:07:06.726 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2021-01-15 19:04:30.160814546 +0000 2021-01-15T19:07:06.726 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2021-01-15T19:07:06.734 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2021-01-15T19:07:06.735 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2021-01-15T19:07:06.735 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 8.4598e-05 s, 6.1 MB/s 2021-01-15T19:07:06.736 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-01-15T19:07:06.792 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-01-15T19:07:06.792 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:06.792 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 112492 Links: 1 2021-01-15T19:07:06.793 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:06.793 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:06.793 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2021-01-15 19:07:02.101358114 +0000 2021-01-15T19:07:06.793 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2021-01-15 19:04:30.495810360 +0000 2021-01-15T19:07:06.793 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2021-01-15 19:04:30.495810360 +0000 2021-01-15T19:07:06.794 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2021-01-15T19:07:06.802 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2021-01-15T19:07:06.802 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2021-01-15T19:07:06.803 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 9.8075e-05 s, 5.2 MB/s 2021-01-15T19:07:06.805 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:06.805 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:06.806 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2021-01-15T19:07:06.821 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-01-15T19:07:06.822 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-01-15T19:07:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-01-15T19:07:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:06.884 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 104363 Links: 1 2021-01-15T19:07:06.885 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:06.885 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:06.885 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2021-01-15 19:06:50.967960677 +0000 2021-01-15T19:07:06.885 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2021-01-15 19:04:19.324719319 +0000 2021-01-15T19:07:06.885 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2021-01-15 19:04:19.324719319 +0000 2021-01-15T19:07:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2021-01-15T19:07:06.894 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2021-01-15T19:07:06.895 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2021-01-15T19:07:06.895 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000258038 s, 2.0 MB/s 2021-01-15T19:07:06.898 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-01-15T19:07:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-01-15T19:07:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:06.965 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 113921 Links: 1 2021-01-15T19:07:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2021-01-15 19:06:50.967960677 +0000 2021-01-15T19:07:06.971 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2021-01-15 19:04:19.643712238 +0000 2021-01-15T19:07:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2021-01-15 19:04:19.643712238 +0000 2021-01-15T19:07:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2021-01-15T19:07:06.978 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2021-01-15T19:07:06.978 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2021-01-15T19:07:06.979 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000206262 s, 2.5 MB/s 2021-01-15T19:07:06.979 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-01-15T19:07:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-01-15T19:07:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:07.049 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 108218 Links: 1 2021-01-15T19:07:07.049 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:07.049 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:07.049 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2021-01-15 19:06:50.967960677 +0000 2021-01-15T19:07:07.050 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2021-01-15 19:04:19.968705024 +0000 2021-01-15T19:07:07.050 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2021-01-15 19:04:19.968705024 +0000 2021-01-15T19:07:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2021-01-15T19:07:07.059 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2021-01-15T19:07:07.060 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2021-01-15T19:07:07.060 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000221441 s, 2.3 MB/s 2021-01-15T19:07:07.062 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-01-15T19:07:07.124 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-01-15T19:07:07.125 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-01-15T19:07:07.125 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 108259 Links: 1 2021-01-15T19:07:07.125 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-01-15T19:07:07.128 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2021-01-15T19:07:07.129 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2021-01-15 19:06:50.967960677 +0000 2021-01-15T19:07:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2021-01-15 19:04:20.295697766 +0000 2021-01-15T19:07:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2021-01-15 19:04:20.295697766 +0000 2021-01-15T19:07:07.132 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2021-01-15T19:07:07.136 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2021-01-15T19:07:07.139 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2021-01-15T19:07:07.141 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000189968 s, 2.7 MB/s 2021-01-15T19:07:07.145 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:07.146 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-01-15T19:07:07.147 INFO:tasks.ceph:Generating config... 2021-01-15T19:07:07.152 INFO:tasks.ceph:[osd] debug monc = 20 2021-01-15T19:07:07.153 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-01-15T19:07:07.154 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2021-01-15T19:07:07.155 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-01-15T19:07:07.156 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-01-15T19:07:07.156 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-01-15T19:07:07.158 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-01-15T19:07:07.160 INFO:tasks.ceph:[osd] debug filestore = 20 2021-01-15T19:07:07.160 INFO:tasks.ceph:[osd] debug journal = 20 2021-01-15T19:07:07.161 INFO:tasks.ceph:[osd] debug ms = 20 2021-01-15T19:07:07.161 INFO:tasks.ceph:[osd] debug osd = 25 2021-01-15T19:07:07.162 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-01-15T19:07:07.162 INFO:tasks.ceph:[osd] filestore odsync write = True 2021-01-15T19:07:07.162 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-01-15T19:07:07.162 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-01-15T19:07:07.163 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-01-15T19:07:07.163 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2021-01-15T19:07:07.163 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2021-01-15T19:07:07.164 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-01-15T19:07:07.164 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2021-01-15T19:07:07.164 INFO:tasks.ceph:[osd] osd max backfills = 2 2021-01-15T19:07:07.165 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2021-01-15T19:07:07.165 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-01-15T19:07:07.165 INFO:tasks.ceph:[osd] osd op queue = debug_random 2021-01-15T19:07:07.165 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2021-01-15T19:07:07.166 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2021-01-15T19:07:07.166 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2021-01-15T19:07:07.167 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2021-01-15T19:07:07.167 INFO:tasks.ceph:[osd] osd snap trim sleep = 0.5 2021-01-15T19:07:07.168 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-01-15T19:07:07.168 INFO:tasks.ceph:[global] mon election default strategy = 1 2021-01-15T19:07:07.168 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-01-15T19:07:07.169 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2021-01-15T19:07:07.169 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-01-15T19:07:07.169 INFO:tasks.ceph:[mgr] debug ms = 1 2021-01-15T19:07:07.169 INFO:tasks.ceph:[mon] debug mon = 20 2021-01-15T19:07:07.170 INFO:tasks.ceph:[mon] debug ms = 1 2021-01-15T19:07:07.170 INFO:tasks.ceph:[mon] debug paxos = 20 2021-01-15T19:07:07.170 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2021-01-15T19:07:07.171 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2021-01-15T19:07:07.171 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2021-01-15T19:07:07.171 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2021-01-15T19:07:07.172 INFO:tasks.ceph:[mon] mon scrub interval = 300 2021-01-15T19:07:07.172 INFO:tasks.ceph:[mon] paxos service trim min = 10 2021-01-15T19:07:07.172 INFO:tasks.ceph:Setting up mon.a... 2021-01-15T19:07:07.173 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2021-01-15T19:07:07.260 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.keyring 2021-01-15T19:07:07.262 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2021-01-15T19:07:07.304 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-01-15T19:07:07.371 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.1'), ('mon.c', '[v2:172.21.15.1:3301,v1:172.21.15.1:6790]'), ('mon.b', '172.21.15.136')] 2021-01-15T19:07:07.372 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.136', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_async_recovery_min_cost': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2021-01-15T19:07:07.375 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:07.375 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-01-15T19:07:07.426 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.1 --addv c '[v2:172.21.15.1:3301,v1:172.21.15.1:6790]' --add b 172.21.15.136 --print /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:07.509 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:07.509 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: generated fsid fe0e1890-043f-40bf-ad11-c5f08349f418 2021-01-15T19:07:07.509 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 0 2021-01-15T19:07:07.510 INFO:teuthology.orchestra.run.smithi001.stdout:fsid fe0e1890-043f-40bf-ad11-c5f08349f418 2021-01-15T19:07:07.510 INFO:teuthology.orchestra.run.smithi001.stdout:last_changed 2021-01-15T19:07:07.508095+0000 2021-01-15T19:07:07.510 INFO:teuthology.orchestra.run.smithi001.stdout:created 2021-01-15T19:07:07.508095+0000 2021-01-15T19:07:07.510 INFO:teuthology.orchestra.run.smithi001.stdout:min_mon_release 0 (unknown) 2021-01-15T19:07:07.511 INFO:teuthology.orchestra.run.smithi001.stdout:election_strategy: 1 2021-01-15T19:07:07.511 INFO:teuthology.orchestra.run.smithi001.stdout:0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2021-01-15T19:07:07.511 INFO:teuthology.orchestra.run.smithi001.stdout:1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2021-01-15T19:07:07.511 INFO:teuthology.orchestra.run.smithi001.stdout:2: [v2:172.21.15.1:3301/0,v1:172.21.15.1:6790/0] mon.c 2021-01-15T19:07:07.511 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2021-01-15T19:07:07.513 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-01-15T19:07:07.568 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fe0e1890-043f-40bf-ad11-c5f08349f418... 2021-01-15T19:07:07.569 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-01-15T19:07:07.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-01-15T19:07:07.653 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2021-01-15T19:07:07.653 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2021-01-15T19:07:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-01-15T19:07:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2021-01-15T19:07:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2021-01-15T19:07:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2021-01-15T19:07:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2021-01-15T19:07:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2021-01-15T19:07:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2021-01-15T19:07:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2021-01-15T19:07:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2021-01-15T19:07:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2021-01-15T19:07:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2021-01-15T19:07:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2021-01-15T19:07:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2021-01-15T19:07:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2021-01-15T19:07:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2021-01-15T19:07:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2021-01-15T19:07:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2021-01-15T19:07:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2021-01-15T19:07:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2021-01-15T19:07:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2021-01-15T19:07:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-01-15T19:07:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2021-01-15T19:07:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2021-01-15T19:07:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-01-15T19:07:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2021-01-15T19:07:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2021-01-15T19:07:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2021-01-15T19:07:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2021-01-15T19:07:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2021-01-15T19:07:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.136" 2021-01-15T19:07:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: mon client directed command retry = 5 2021-01-15T19:07:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: mon election default strategy = 1 2021-01-15T19:07:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: ms inject socket failures = 5000 2021-01-15T19:07:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: osd_async_recovery_min_cost = 1 2021-01-15T19:07:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = fe0e1890-043f-40bf-ad11-c5f08349f418 2021-01-15T19:07:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.664 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2021-01-15T19:07:07.666 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2021-01-15T19:07:07.666 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.667 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2021-01-15T19:07:07.667 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 120 2021-01-15T19:07:07.668 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.668 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2021-01-15T19:07:07.669 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2021-01-15T19:07:07.669 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2021-01-15T19:07:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2021-01-15T19:07:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2021-01-15T19:07:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2021-01-15T19:07:07.672 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2021-01-15T19:07:07.672 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.673 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2021-01-15T19:07:07.673 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.674 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2021-01-15T19:07:07.674 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2021-01-15T19:07:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2021-01-15T19:07:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2021-01-15T19:07:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2021-01-15T19:07:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2021-01-15T19:07:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: debug monc = 20 2021-01-15T19:07:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2021-01-15T19:07:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore compression algorithm = lz4 2021-01-15T19:07:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore compression mode = aggressive 2021-01-15T19:07:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2021-01-15T19:07:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2021-01-15T19:07:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2021-01-15T19:07:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: debug filestore = 20 2021-01-15T19:07:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: debug journal = 20 2021-01-15T19:07:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 20 2021-01-15T19:07:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 25 2021-01-15T19:07:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2021-01-15T19:07:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: filestore odsync write = True 2021-01-15T19:07:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2021-01-15T19:07:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2021-01-15T19:07:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2021-01-15T19:07:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify cached snaps = True 2021-01-15T19:07:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify missing on start = True 2021-01-15T19:07:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2021-01-15T19:07:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: osd heartbeat use min delay socket = True 2021-01-15T19:07:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: osd max backfills = 2 2021-01-15T19:07:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: osd max markdown count = 1000 2021-01-15T19:07:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2021-01-15T19:07:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: osd op queue = debug_random 2021-01-15T19:07:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: osd op queue cut off = debug_random 2021-01-15T19:07:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub min interval = 60 2021-01-15T19:07:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: osd shutdown pgref assert = True 2021-01-15T19:07:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: osd snap trim sleep = 0.5 2021-01-15T19:07:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.682 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2021-01-15T19:07:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2021-01-15T19:07:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2021-01-15T19:07:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2021-01-15T19:07:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2021-01-15T19:07:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2021-01-15T19:07:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2021-01-15T19:07:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2021-01-15T19:07:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.684 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2021-01-15T19:07:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2021-01-15T19:07:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2021-01-15T19:07:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2021-01-15T19:07:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2021-01-15T19:07:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2021-01-15T19:07:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2021-01-15T19:07:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2021-01-15T19:07:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2021-01-15T19:07:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2021-01-15T19:07:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2021-01-15T19:07:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: mon min osdmap epochs = 50 2021-01-15T19:07:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: mon osdmap full prune interval = 2 2021-01-15T19:07:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: mon osdmap full prune min = 15 2021-01-15T19:07:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: mon osdmap full prune txsize = 2 2021-01-15T19:07:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: mon scrub interval = 300 2021-01-15T19:07:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: paxos service trim min = 10 2021-01-15T19:07:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:07.688 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2021-01-15T19:07:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2021-01-15T19:07:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2021-01-15T19:07:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2021-01-15T19:07:07.689 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-01-15T19:07:07.689 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-01-15T19:07:07.689 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2021-01-15T19:07:07.689 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2021-01-15T19:07:07.689 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2021-01-15T19:07:07.691 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2021-01-15T19:07:07.691 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2021-01-15T19:07:07.691 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-01-15T19:07:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2021-01-15T19:07:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2021-01-15T19:07:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2021-01-15T19:07:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2021-01-15T19:07:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2021-01-15T19:07:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2021-01-15T19:07:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2021-01-15T19:07:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2021-01-15T19:07:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2021-01-15T19:07:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2021-01-15T19:07:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2021-01-15T19:07:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2021-01-15T19:07:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2021-01-15T19:07:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2021-01-15T19:07:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2021-01-15T19:07:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2021-01-15T19:07:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2021-01-15T19:07:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2021-01-15T19:07:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2021-01-15T19:07:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-01-15T19:07:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2021-01-15T19:07:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2021-01-15T19:07:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-01-15T19:07:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2021-01-15T19:07:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2021-01-15T19:07:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2021-01-15T19:07:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2021-01-15T19:07:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2021-01-15T19:07:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.136" 2021-01-15T19:07:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: mon client directed command retry = 5 2021-01-15T19:07:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: mon election default strategy = 1 2021-01-15T19:07:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject socket failures = 5000 2021-01-15T19:07:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: osd_async_recovery_min_cost = 1 2021-01-15T19:07:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = fe0e1890-043f-40bf-ad11-c5f08349f418 2021-01-15T19:07:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.701 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2021-01-15T19:07:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2021-01-15T19:07:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2021-01-15T19:07:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 120 2021-01-15T19:07:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2021-01-15T19:07:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2021-01-15T19:07:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2021-01-15T19:07:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2021-01-15T19:07:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2021-01-15T19:07:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2021-01-15T19:07:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2021-01-15T19:07:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2021-01-15T19:07:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2021-01-15T19:07:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2021-01-15T19:07:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2021-01-15T19:07:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2021-01-15T19:07:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2021-01-15T19:07:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2021-01-15T19:07:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: debug monc = 20 2021-01-15T19:07:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2021-01-15T19:07:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression algorithm = lz4 2021-01-15T19:07:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore compression mode = aggressive 2021-01-15T19:07:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2021-01-15T19:07:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2021-01-15T19:07:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2021-01-15T19:07:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: debug filestore = 20 2021-01-15T19:07:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: debug journal = 20 2021-01-15T19:07:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 20 2021-01-15T19:07:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 25 2021-01-15T19:07:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2021-01-15T19:07:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: filestore odsync write = True 2021-01-15T19:07:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2021-01-15T19:07:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2021-01-15T19:07:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2021-01-15T19:07:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify cached snaps = True 2021-01-15T19:07:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify missing on start = True 2021-01-15T19:07:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2021-01-15T19:07:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: osd heartbeat use min delay socket = True 2021-01-15T19:07:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: osd max backfills = 2 2021-01-15T19:07:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: osd max markdown count = 1000 2021-01-15T19:07:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2021-01-15T19:07:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue = debug_random 2021-01-15T19:07:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue cut off = debug_random 2021-01-15T19:07:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub min interval = 60 2021-01-15T19:07:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: osd shutdown pgref assert = True 2021-01-15T19:07:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: osd snap trim sleep = 0.5 2021-01-15T19:07:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.713 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2021-01-15T19:07:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2021-01-15T19:07:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2021-01-15T19:07:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2021-01-15T19:07:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2021-01-15T19:07:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2021-01-15T19:07:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2021-01-15T19:07:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2021-01-15T19:07:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.715 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2021-01-15T19:07:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2021-01-15T19:07:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2021-01-15T19:07:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2021-01-15T19:07:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2021-01-15T19:07:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2021-01-15T19:07:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2021-01-15T19:07:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2021-01-15T19:07:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2021-01-15T19:07:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2021-01-15T19:07:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2021-01-15T19:07:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: mon min osdmap epochs = 50 2021-01-15T19:07:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: mon osdmap full prune interval = 2 2021-01-15T19:07:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: mon osdmap full prune min = 15 2021-01-15T19:07:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: mon osdmap full prune txsize = 2 2021-01-15T19:07:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: mon scrub interval = 300 2021-01-15T19:07:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: paxos service trim min = 10 2021-01-15T19:07:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-01-15T19:07:07.718 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2021-01-15T19:07:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2021-01-15T19:07:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2021-01-15T19:07:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2021-01-15T19:07:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-01-15T19:07:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-01-15T19:07:07.720 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2021-01-15T19:07:07.720 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2021-01-15T19:07:07.720 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2021-01-15T19:07:07.720 INFO:tasks.ceph:Creating admin key on mon.a... 2021-01-15T19:07:07.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2021-01-15T19:07:07.783 INFO:tasks.ceph:Copying monmap to all nodes... 2021-01-15T19:07:07.783 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:07.784 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2021-01-15T19:07:07.799 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:07.799 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2021-01-15T19:07:07.853 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:07:07.854 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:07.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.keyring 2021-01-15T19:07:07.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-01-15T19:07:07.931 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:07.931 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:07.987 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T19:07:07.988 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:07.988 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2021-01-15T19:07:07.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-01-15T19:07:08.032 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:08.032 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:08.088 INFO:tasks.ceph:Setting up mon nodes... 2021-01-15T19:07:08.089 INFO:tasks.ceph:Setting up mgr nodes... 2021-01-15T19:07:08.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2021-01-15T19:07:08.141 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2021-01-15T19:07:08.143 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2021-01-15T19:07:08.220 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2021-01-15T19:07:08.223 INFO:tasks.ceph:Setting up mds nodes... 2021-01-15T19:07:08.223 INFO:tasks.ceph_client:Setting up client nodes... 2021-01-15T19:07:08.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-01-15T19:07:08.264 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.client.0.keyring 2021-01-15T19:07:08.277 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-01-15T19:07:08.319 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.client.1.keyring 2021-01-15T19:07:08.334 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-01-15T19:07:08.334 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-01-15T19:07:08.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2021-01-15T19:07:08.361 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:08.362 INFO:tasks.ceph:role: osd.0 2021-01-15T19:07:08.362 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:07:08.362 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-01-15T19:07:08.428 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:08.429 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:08.429 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:08.429 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 2021-01-15T19:07:08.430 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:08.430 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:08.430 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:08.430 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:08.430 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:08.431 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:08.452 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2021-01-15T19:07:08.453 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2021-01-15T19:07:08.454 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2021-01-15T19:07:08.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2021-01-15T19:07:08.603 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2021-01-15T19:07:08.670 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:08.671 INFO:tasks.ceph:role: osd.1 2021-01-15T19:07:08.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:07:08.672 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-01-15T19:07:08.736 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 2021-01-15T19:07:08.737 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:08.737 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:08.737 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:08.738 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:08.738 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:08.758 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2021-01-15T19:07:08.759 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2021-01-15T19:07:08.761 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2021-01-15T19:07:08.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2021-01-15T19:07:08.911 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2021-01-15T19:07:08.977 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:08.978 INFO:tasks.ceph:role: osd.2 2021-01-15T19:07:08.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:07:08.979 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-01-15T19:07:09.043 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:09.043 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:09.044 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:09.044 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 2021-01-15T19:07:09.044 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:09.044 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:09.044 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:09.045 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:09.045 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:09.045 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:09.066 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2021-01-15T19:07:09.068 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2021-01-15T19:07:09.068 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2021-01-15T19:07:09.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2021-01-15T19:07:09.215 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2021-01-15T19:07:09.283 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:09.284 INFO:tasks.ceph:role: osd.3 2021-01-15T19:07:09.284 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:07:09.285 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-01-15T19:07:09.349 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:09.350 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:09.350 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:09.350 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 2021-01-15T19:07:09.350 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:09.351 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:09.351 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:09.351 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:09.351 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:09.351 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:09.372 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2021-01-15T19:07:09.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2021-01-15T19:07:09.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2021-01-15T19:07:09.452 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2021-01-15T19:07:09.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:09.623 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:09.621+0000 7efdfe9ea080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2021-01-15T19:07:09.623 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:09.621+0000 7efdfe9ea080 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2021-01-15T19:07:09.624 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:09.621+0000 7efdfe9ea080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-01-15T19:07:09.624 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:09.621+0000 7efdfe9ea080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-01-15T19:07:09.624 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:09.621+0000 7efdfe9ea080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-01-15T19:07:09.624 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:09.621+0000 7efdfe9ea080 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2021-01-15T19:07:12.467 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-01-15T19:07:12.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:12.589 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:12.587+0000 7fa43f949080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2021-01-15T19:07:12.590 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:12.587+0000 7fa43f949080 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2021-01-15T19:07:12.590 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:12.587+0000 7fa43f949080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-01-15T19:07:12.590 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:12.587+0000 7fa43f949080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-01-15T19:07:12.590 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:12.587+0000 7fa43f949080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-01-15T19:07:12.590 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:12.588+0000 7fa43f949080 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2021-01-15T19:07:15.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-01-15T19:07:15.467 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:15.554+0000 7f82ecd84080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2021-01-15T19:07:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:15.554+0000 7f82ecd84080 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2021-01-15T19:07:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:15.554+0000 7f82ecd84080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-01-15T19:07:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:15.554+0000 7f82ecd84080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-01-15T19:07:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:15.554+0000 7f82ecd84080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-01-15T19:07:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:15.554+0000 7f82ecd84080 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2021-01-15T19:07:18.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-01-15T19:07:18.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:18.518+0000 7fa84026c080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2021-01-15T19:07:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:18.518+0000 7fa84026c080 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2021-01-15T19:07:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:18.518+0000 7fa84026c080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-01-15T19:07:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:18.518+0000 7fa84026c080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-01-15T19:07:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:18.518+0000 7fa84026c080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-01-15T19:07:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2021-01-15T19:07:18.518+0000 7fa84026c080 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2021-01-15T19:07:21.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-01-15T19:07:21.394 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2021-01-15T19:07:21.424 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:21.425 INFO:tasks.ceph:role: osd.4 2021-01-15T19:07:21.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T19:07:21.425 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-01-15T19:07:21.500 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:21.501 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:21.501 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:21.502 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2021-01-15T19:07:21.502 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:21.502 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:21.502 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:21.502 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:21.503 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:21.503 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:21.924 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2021-01-15T19:07:21.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2021-01-15T19:07:21.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2021-01-15T19:07:22.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2021-01-15T19:07:22.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2021-01-15T19:07:22.139 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:22.140 INFO:tasks.ceph:role: osd.5 2021-01-15T19:07:22.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T19:07:22.141 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-01-15T19:07:22.213 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:22.214 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:22.215 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2021-01-15T19:07:22.216 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:22.216 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:22.216 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:22.217 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:22.217 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:22.217 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:22.579 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2021-01-15T19:07:22.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2021-01-15T19:07:22.581 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2021-01-15T19:07:22.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2021-01-15T19:07:22.732 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2021-01-15T19:07:22.799 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:22.799 INFO:tasks.ceph:role: osd.6 2021-01-15T19:07:22.799 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T19:07:22.800 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-01-15T19:07:22.871 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2021-01-15T19:07:22.872 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:22.872 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:22.873 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:22.873 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:22.873 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:23.287 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2021-01-15T19:07:23.288 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2021-01-15T19:07:23.289 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2021-01-15T19:07:23.383 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2021-01-15T19:07:23.411 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2021-01-15T19:07:23.475 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-01-15T19:07:23.475 INFO:tasks.ceph:role: osd.7 2021-01-15T19:07:23.475 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T19:07:23.476 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-01-15T19:07:23.546 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2021-01-15T19:07:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2021-01-15T19:07:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-01-15T19:07:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2021-01-15T19:07:23.547 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-01-15T19:07:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2021-01-15T19:07:23.548 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-01-15T19:07:23.548 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-01-15T19:07:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-01-15T19:07:23.548 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-01-15T19:07:23.973 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2021-01-15T19:07:23.974 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2021-01-15T19:07:23.975 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2021-01-15T19:07:24.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2021-01-15T19:07:24.101 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:24.194 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:24.193+0000 7fcd3a188080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2021-01-15T19:07:24.196 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:24.194+0000 7fcd3a188080 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2021-01-15T19:07:24.196 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:24.194+0000 7fcd3a188080 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-01-15T19:07:24.196 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:24.194+0000 7fcd3a188080 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-01-15T19:07:24.196 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:24.194+0000 7fcd3a188080 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-01-15T19:07:24.197 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:24.195+0000 7fcd3a188080 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2021-01-15T19:07:27.043 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-01-15T19:07:27.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:27.159 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:27.157+0000 7f22c282c080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2021-01-15T19:07:27.159 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:27.157+0000 7f22c282c080 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2021-01-15T19:07:27.159 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:27.157+0000 7f22c282c080 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-01-15T19:07:27.160 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:27.157+0000 7f22c282c080 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-01-15T19:07:27.160 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:27.157+0000 7f22c282c080 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-01-15T19:07:27.160 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:27.157+0000 7f22c282c080 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2021-01-15T19:07:30.001 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-01-15T19:07:30.030 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:30.115 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:30.113+0000 7f12e6197080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2021-01-15T19:07:30.116 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:30.114+0000 7f12e6197080 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2021-01-15T19:07:30.116 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:30.114+0000 7f12e6197080 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-01-15T19:07:30.116 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:30.114+0000 7f12e6197080 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-01-15T19:07:30.116 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:30.114+0000 7f12e6197080 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-01-15T19:07:30.117 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:30.114+0000 7f12e6197080 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2021-01-15T19:07:32.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-01-15T19:07:32.987 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:33.075 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:33.072+0000 7fe842ed9080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2021-01-15T19:07:33.076 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:33.072+0000 7fe842ed9080 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2021-01-15T19:07:33.076 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:33.072+0000 7fe842ed9080 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-01-15T19:07:33.077 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:33.072+0000 7fe842ed9080 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-01-15T19:07:33.077 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:33.072+0000 7fe842ed9080 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-01-15T19:07:33.077 INFO:teuthology.orchestra.run.smithi136.stderr:2021-01-15T19:07:33.073+0000 7fe842ed9080 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2021-01-15T19:07:35.921 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-01-15T19:07:35.949 INFO:tasks.ceph:Reading keys from all nodes... 2021-01-15T19:07:35.949 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:35.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2021-01-15T19:07:35.980 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:36.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2021-01-15T19:07:36.273 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:36.274 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2021-01-15T19:07:36.343 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:36.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2021-01-15T19:07:36.411 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:36.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2021-01-15T19:07:36.479 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:36.480 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2021-01-15T19:07:36.509 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:36.510 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2021-01-15T19:07:36.578 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:36.579 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2021-01-15T19:07:36.646 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:36.646 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2021-01-15T19:07:36.712 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:36.713 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2021-01-15T19:07:36.778 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:36.779 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2021-01-15T19:07:36.796 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:36.797 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2021-01-15T19:07:36.833 INFO:tasks.ceph:Adding keys to all mons... 2021-01-15T19:07:36.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo tee -a /etc/ceph/ceph.keyring 2021-01-15T19:07:36.839 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2021-01-15T19:07:36.877 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.y] 2021-01-15T19:07:36.877 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDc5wFgrhpUCBAA9gL0Uu+XyhBK9eLiDkKrxg== 2021-01-15T19:07:36.878 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.0] 2021-01-15T19:07:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDd5wFgVj4WJRAAgPr9yMDdFLb0vj/sOL6wHg== 2021-01-15T19:07:36.878 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.1] 2021-01-15T19:07:36.878 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDg5wFgYPkTIxAAhEHA3IbcywXdpgxMiSArlg== 2021-01-15T19:07:36.879 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.2] 2021-01-15T19:07:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDj5wFgCxYYIRAAVYOIeO1i7LT24AuTJMxDlQ== 2021-01-15T19:07:36.879 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.3] 2021-01-15T19:07:36.879 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDm5wFg5VHwHhAAVhNb+oNW6KSy3dJX5uoaSA== 2021-01-15T19:07:36.880 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.x] 2021-01-15T19:07:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDc5wFgb84bDRAAf5rTp/faCCWlnmJB12xTjg== 2021-01-15T19:07:36.880 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.4] 2021-01-15T19:07:36.880 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDs5wFgLV+RCxAAaW1Hd1UauxwJwew+vFpHTQ== 2021-01-15T19:07:36.881 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.5] 2021-01-15T19:07:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDv5wFghZtoCRAAX3RBCv2dlKGEu5VFqPhDww== 2021-01-15T19:07:36.881 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.6] 2021-01-15T19:07:36.881 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDy5wFgJybXBhAAfdwv5irlfXt2O3KINtihhA== 2021-01-15T19:07:36.882 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.7] 2021-01-15T19:07:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQD15wFgBslfBBAAZOK2UYhT7onmb/wb7A+Cqw== 2021-01-15T19:07:36.882 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2021-01-15T19:07:36.882 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDc5wFgKpKvDxAACK0jHsGYAXcjvIAAX02DFA== 2021-01-15T19:07:36.882 INFO:teuthology.orchestra.run.smithi001.stdout:[client.1] 2021-01-15T19:07:36.883 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDc5wFgtPcBExAAcMjcxO+TWEqULe4aIVaMkg== 2021-01-15T19:07:36.898 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2021-01-15T19:07:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDc5wFgrhpUCBAA9gL0Uu+XyhBK9eLiDkKrxg== 2021-01-15T19:07:36.900 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2021-01-15T19:07:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDd5wFgVj4WJRAAgPr9yMDdFLb0vj/sOL6wHg== 2021-01-15T19:07:36.901 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2021-01-15T19:07:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDg5wFgYPkTIxAAhEHA3IbcywXdpgxMiSArlg== 2021-01-15T19:07:36.901 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2021-01-15T19:07:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDj5wFgCxYYIRAAVYOIeO1i7LT24AuTJMxDlQ== 2021-01-15T19:07:36.902 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2021-01-15T19:07:36.902 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDm5wFg5VHwHhAAVhNb+oNW6KSy3dJX5uoaSA== 2021-01-15T19:07:36.902 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2021-01-15T19:07:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDc5wFgb84bDRAAf5rTp/faCCWlnmJB12xTjg== 2021-01-15T19:07:36.903 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2021-01-15T19:07:36.904 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDs5wFgLV+RCxAAaW1Hd1UauxwJwew+vFpHTQ== 2021-01-15T19:07:36.905 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2021-01-15T19:07:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDv5wFghZtoCRAAX3RBCv2dlKGEu5VFqPhDww== 2021-01-15T19:07:36.905 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2021-01-15T19:07:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDy5wFgJybXBhAAfdwv5irlfXt2O3KINtihhA== 2021-01-15T19:07:36.906 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2021-01-15T19:07:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQD15wFgBslfBBAAZOK2UYhT7onmb/wb7A+Cqw== 2021-01-15T19:07:36.906 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2021-01-15T19:07:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDc5wFgKpKvDxAACK0jHsGYAXcjvIAAX02DFA== 2021-01-15T19:07:36.906 INFO:teuthology.orchestra.run.smithi136.stdout:[client.1] 2021-01-15T19:07:36.907 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQDc5wFgtPcBExAAcMjcxO+TWEqULe4aIVaMkg== 2021-01-15T19:07:36.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-15T19:07:36.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-15T19:07:36.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:36.998 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.049 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.097 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.146 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-15T19:07:37.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-01-15T19:07:37.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.243 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.291 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.337 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.389 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-01-15T19:07:37.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-01-15T19:07:37.437 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-01-15T19:07:37.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-01-15T19:07:37.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-01-15T19:07:37.537 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-01-15T19:07:37.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2021-01-15T19:07:37.564 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-01-15T19:07:37.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2021-01-15T19:07:37.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2021-01-15T19:07:37.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-01-15T19:07:37.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2021-01-15T19:07:37.871 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2021-01-15T19:07:37.896 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-01-15T19:07:37.990 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2021-01-15T19:07:38.015 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:38.017 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-01-15T19:07:38.068 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2021-01-15T19:07:38.069 INFO:tasks.ceph.mon.a:Restarting daemon 2021-01-15T19:07:38.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2021-01-15T19:07:38.072 INFO:tasks.ceph.mon.a:Started 2021-01-15T19:07:38.072 INFO:tasks.ceph.mon.c:Restarting daemon 2021-01-15T19:07:38.073 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2021-01-15T19:07:38.075 INFO:tasks.ceph.mon.c:Started 2021-01-15T19:07:38.075 INFO:tasks.ceph.mon.b:Restarting daemon 2021-01-15T19:07:38.076 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2021-01-15T19:07:38.111 INFO:tasks.ceph.mon.b:Started 2021-01-15T19:07:38.111 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2021-01-15T19:07:38.112 INFO:tasks.ceph.mgr.y:Restarting daemon 2021-01-15T19:07:38.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2021-01-15T19:07:38.114 INFO:tasks.ceph.mgr.y:Started 2021-01-15T19:07:38.115 INFO:tasks.ceph.mgr.x:Restarting daemon 2021-01-15T19:07:38.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2021-01-15T19:07:38.117 INFO:tasks.ceph.mgr.x:Started 2021-01-15T19:07:38.118 INFO:tasks.ceph:Setting crush tunables to default 2021-01-15T19:07:38.118 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd crush tunables default 2021-01-15T19:07:43.509 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2021-01-15T19:07:43.520 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2021-01-15T19:07:43.521 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:43.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2021-01-15T19:07:43.549 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:43.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2021-01-15T19:07:43.631 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:43.632 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2021-01-15T19:07:43.699 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2021-01-15T19:07:43.700 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2021-01-15T19:07:43.766 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:43.766 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2021-01-15T19:07:43.797 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:43.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2021-01-15T19:07:43.865 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:43.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2021-01-15T19:07:43.932 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2021-01-15T19:07:43.933 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2021-01-15T19:07:43.999 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new e5b26f89-5c99-4764-9183-f3bb27c118f5 0 2021-01-15T19:07:44.259 INFO:teuthology.orchestra.run.smithi136.stdout:0 2021-01-15T19:07:44.271 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 99c0d455-1bd6-43aa-bfec-b6bddf87a35e 1 2021-01-15T19:07:44.494 INFO:teuthology.orchestra.run.smithi136.stdout:1 2021-01-15T19:07:44.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 71fe6c4b-3d73-4e07-b49d-98596ca7d9da 2 2021-01-15T19:07:44.726 INFO:teuthology.orchestra.run.smithi136.stdout:2 2021-01-15T19:07:44.738 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 8a6b018a-3a48-4a01-8526-a386f62ecb36 3 2021-01-15T19:07:44.957 INFO:teuthology.orchestra.run.smithi136.stdout:3 2021-01-15T19:07:44.972 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new ff089622-e642-4bd0-901c-60efdb87a549 4 2021-01-15T19:07:45.189 INFO:teuthology.orchestra.run.smithi136.stdout:4 2021-01-15T19:07:45.200 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 3c57c228-dfaa-4e33-bded-27da0048d141 5 2021-01-15T19:07:45.452 INFO:teuthology.orchestra.run.smithi136.stdout:5 2021-01-15T19:07:45.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 201f889f-dac1-4a26-a285-7a0ec21c3646 6 2021-01-15T19:07:45.685 INFO:teuthology.orchestra.run.smithi136.stdout:6 2021-01-15T19:07:45.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 79ffd413-a9d3-4180-9691-d834d89c3f2e 7 2021-01-15T19:07:45.922 INFO:teuthology.orchestra.run.smithi136.stdout:7 2021-01-15T19:07:45.934 INFO:tasks.ceph.osd.0:Restarting daemon 2021-01-15T19:07:45.935 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2021-01-15T19:07:45.938 INFO:tasks.ceph.osd.0:Started 2021-01-15T19:07:45.938 INFO:tasks.ceph.osd.1:Restarting daemon 2021-01-15T19:07:45.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2021-01-15T19:07:45.944 INFO:tasks.ceph.osd.1:Started 2021-01-15T19:07:45.944 INFO:tasks.ceph.osd.2:Restarting daemon 2021-01-15T19:07:45.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2021-01-15T19:07:45.947 INFO:tasks.ceph.osd.2:Started 2021-01-15T19:07:45.947 INFO:tasks.ceph.osd.3:Restarting daemon 2021-01-15T19:07:45.948 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2021-01-15T19:07:45.950 INFO:tasks.ceph.osd.3:Started 2021-01-15T19:07:45.951 INFO:tasks.ceph.osd.4:Restarting daemon 2021-01-15T19:07:45.951 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2021-01-15T19:07:45.953 INFO:tasks.ceph.osd.4:Started 2021-01-15T19:07:45.954 INFO:tasks.ceph.osd.5:Restarting daemon 2021-01-15T19:07:45.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2021-01-15T19:07:45.956 INFO:tasks.ceph.osd.5:Started 2021-01-15T19:07:45.956 INFO:tasks.ceph.osd.6:Restarting daemon 2021-01-15T19:07:45.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-01-15T19:07:45.959 INFO:tasks.ceph.osd.6:Started 2021-01-15T19:07:45.959 INFO:tasks.ceph.osd.7:Restarting daemon 2021-01-15T19:07:45.959 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2021-01-15T19:07:45.961 INFO:tasks.ceph.osd.7:Started 2021-01-15T19:07:45.962 INFO:tasks.ceph:Waiting for OSDs to come up 2021-01-15T19:07:46.535 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:07:46.533+0000 7fb6d6981080 -1 Falling back to public interface 2021-01-15T19:07:46.550 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:07:46.547+0000 7f0cdd523080 -1 Falling back to public interface 2021-01-15T19:07:46.551 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:07:46.548+0000 7fed65a79080 -1 Falling back to public interface 2021-01-15T19:07:46.553 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:07:46.551+0000 7ff293edd080 -1 Falling back to public interface 2021-01-15T19:07:46.553 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:07:46.552+0000 7f0f713cc080 -1 Falling back to public interface 2021-01-15T19:07:46.578 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:07:46.572+0000 7ff447b9b080 -1 Falling back to public interface 2021-01-15T19:07:46.579 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:07:46.572+0000 7f83e08d7080 -1 Falling back to public interface 2021-01-15T19:07:46.582 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:07:46.580+0000 7f579e20c080 -1 Falling back to public interface 2021-01-15T19:07:46.768 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-01-15T19:07:46.985 INFO:teuthology.misc.health.smithi001.stdout: 2021-01-15T19:07:46.985 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":10,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:07:45.917848+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:07:46.997 DEBUG:teuthology.misc:0 of 8 OSDs are up 2021-01-15T19:07:47.854 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:07:47.853+0000 7fb6d6981080 -1 osd.0 0 log_to_monitors {default=true} 2021-01-15T19:07:47.886 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:07:47.884+0000 7fed65a79080 -1 osd.1 0 log_to_monitors {default=true} 2021-01-15T19:07:47.886 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:07:47.885+0000 7f0cdd523080 -1 osd.4 0 log_to_monitors {default=true} 2021-01-15T19:07:47.894 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:07:47.892+0000 7ff293edd080 -1 osd.6 0 log_to_monitors {default=true} 2021-01-15T19:07:47.902 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:07:47.900+0000 7f0f713cc080 -1 osd.5 0 log_to_monitors {default=true} 2021-01-15T19:07:47.904 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:07:47.902+0000 7f83e08d7080 -1 osd.3 0 log_to_monitors {default=true} 2021-01-15T19:07:47.920 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:07:47.919+0000 7ff447b9b080 -1 osd.2 0 log_to_monitors {default=true} 2021-01-15T19:07:47.926 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:07:47.924+0000 7f579e20c080 -1 osd.7 0 log_to_monitors {default=true} 2021-01-15T19:07:49.517 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:07:49.515+0000 7f579499e700 -1 osd.7 0 waiting for initial osdmap 2021-01-15T19:07:49.518 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:07:49.516+0000 7f0f67b5e700 -1 osd.5 0 waiting for initial osdmap 2021-01-15T19:07:49.518 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:07:49.516+0000 7fb6cd113700 -1 osd.0 0 waiting for initial osdmap 2021-01-15T19:07:49.519 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:07:49.517+0000 7ff28a66f700 -1 osd.6 0 waiting for initial osdmap 2021-01-15T19:07:49.519 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:07:49.517+0000 7fed5c20b700 -1 osd.1 0 waiting for initial osdmap 2021-01-15T19:07:49.520 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:07:49.518+0000 7f0cd3cb5700 -1 osd.4 0 waiting for initial osdmap 2021-01-15T19:07:49.521 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:07:49.518+0000 7ff43e32d700 -1 osd.2 0 waiting for initial osdmap 2021-01-15T19:07:49.522 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:07:49.517+0000 7f83d7069700 -1 osd.3 0 waiting for initial osdmap 2021-01-15T19:07:53.805 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-01-15T19:07:54.127 INFO:teuthology.misc.health.smithi001.stdout: 2021-01-15T19:07:54.127 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":15,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:07:52.527778+0000","last_up_change":"2021-01-15T19:07:50.512119+0000","last_in_change":"2021-01-15T19:07:50.512119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists","up"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists","up"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":34831},{"type":"v1","addr":"172.21.15.136:6810","nonce":34831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":34831},{"type":"v1","addr":"172.21.15.136:6814","nonce":34831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":34831},{"type":"v1","addr":"172.21.15.136:6822","nonce":34831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":34831},{"type":"v1","addr":"172.21.15.136:6818","nonce":34831}]},"public_addr":"172.21.15.136:6810/34831","cluster_addr":"172.21.15.136:6814/34831","heartbeat_back_addr":"172.21.15.136:6822/34831","heartbeat_front_addr":"172.21.15.136:6818/34831","state":["exists","up"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.903933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.899879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.904316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.941511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.851510+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.879200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.918755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.914415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:07:54.133 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-01-15T19:07:54.133 INFO:tasks.ceph:Creating RBD pool 2021-01-15T19:07:54.134 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd pool create rbd 8 2021-01-15T19:07:55.173 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'rbd' created 2021-01-15T19:07:55.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2021-01-15T19:07:56.181 INFO:teuthology.orchestra.run.smithi001.stderr:enabled application 'rbd' on pool 'rbd' 2021-01-15T19:07:56.196 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2021-01-15T19:07:56.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-15T19:07:56.197 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-01-15T19:07:56.516 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:56.517 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:07:56.167459+0000","last_up_change":"2021-01-15T19:07:50.512119+0000","last_in_change":"2021-01-15T19:07:50.512119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-15T19:07:54.484638+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists","up"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists","up"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":34831},{"type":"v1","addr":"172.21.15.136:6810","nonce":34831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":34831},{"type":"v1","addr":"172.21.15.136:6814","nonce":34831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":34831},{"type":"v1","addr":"172.21.15.136:6822","nonce":34831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":34831},{"type":"v1","addr":"172.21.15.136:6818","nonce":34831}]},"public_addr":"172.21.15.136:6810/34831","cluster_addr":"172.21.15.136:6814/34831","heartbeat_back_addr":"172.21.15.136:6822/34831","heartbeat_front_addr":"172.21.15.136:6818/34831","state":["exists","up"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.903933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.899879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.904316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.941511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.851510+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.879200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.918755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.914415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:07:56.529 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-01-15T19:07:56.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-01-15T19:07:56.847 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2021-01-15T19:07:56.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-01-15T19:07:57.179 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 8 2021-01-15T19:07:57.192 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-01-15T19:07:57.192 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2021-01-15T19:07:57.193 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2021-01-15T19:07:57.551 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:57.564 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":34624},{"type":"v1","addr":"172.21.15.1:6833","nonce":34624}]},"active_addr":"172.21.15.1:6833/34624","active_change":"2021-01-15T19:07:47.498175+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4117,"name":"x","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":469065039}]},{"addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":4191424063}]},{"addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3722836723}]}]}} 2021-01-15T19:07:57.568 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2021-01-15T19:07:57.569 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2021-01-15T19:07:57.569 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-15T19:07:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:57.880 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:07:57.172007+0000","last_up_change":"2021-01-15T19:07:50.512119+0000","last_in_change":"2021-01-15T19:07:50.512119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-15T19:07:54.484638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists","up"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists","up"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":34831},{"type":"v1","addr":"172.21.15.136:6810","nonce":34831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":34831},{"type":"v1","addr":"172.21.15.136:6814","nonce":34831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":34831},{"type":"v1","addr":"172.21.15.136:6822","nonce":34831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":34831},{"type":"v1","addr":"172.21.15.136:6818","nonce":34831}]},"public_addr":"172.21.15.136:6810/34831","cluster_addr":"172.21.15.136:6814/34831","heartbeat_back_addr":"172.21.15.136:6822/34831","heartbeat_front_addr":"172.21.15.136:6818/34831","state":["exists","up"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.903933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.899879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.904316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.941511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.851510+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.879200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.918755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.914415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:07:57.893 INFO:tasks.ceph.ceph_manager.ceph:all up! 2021-01-15T19:07:57.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-15T19:07:58.209 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:07:58.209 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":18,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:07:57.172007+0000","last_up_change":"2021-01-15T19:07:50.512119+0000","last_in_change":"2021-01-15T19:07:50.512119+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-15T19:07:54.484638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists","up"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists","up"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":34831},{"type":"v1","addr":"172.21.15.136:6810","nonce":34831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":34831},{"type":"v1","addr":"172.21.15.136:6814","nonce":34831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":34831},{"type":"v1","addr":"172.21.15.136:6822","nonce":34831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":34831},{"type":"v1","addr":"172.21.15.136:6818","nonce":34831}]},"public_addr":"172.21.15.136:6810/34831","cluster_addr":"172.21.15.136:6814/34831","heartbeat_back_addr":"172.21.15.136:6822/34831","heartbeat_front_addr":"172.21.15.136:6818/34831","state":["exists","up"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.903933+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.899879+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.904316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.941511+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.851510+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.879200+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.918755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:07:48.914415+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:07:58.222 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2021-01-15T19:07:58.382 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:07:58.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2021-01-15T19:07:58.542 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:07:58.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2021-01-15T19:07:58.703 INFO:teuthology.orchestra.run.smithi001.stdout:55834574852 2021-01-15T19:07:58.704 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2021-01-15T19:07:58.863 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:07:58.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2021-01-15T19:07:59.025 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:07:59.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2021-01-15T19:07:59.187 INFO:teuthology.orchestra.run.smithi001.stdout:55834574852 2021-01-15T19:07:59.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2021-01-15T19:07:59.347 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:07:59.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2021-01-15T19:07:59.508 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:07:59.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-01-15T19:07:59.823 INFO:teuthology.orchestra.run.smithi001.stdout:55834574850 2021-01-15T19:07:59.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2021-01-15T19:08:00.837 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-01-15T19:08:01.165 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:08:01.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2021-01-15T19:08:01.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2021-01-15T19:08:01.494 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:08:01.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2021-01-15T19:08:01.507 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2021-01-15T19:08:01.823 INFO:teuthology.orchestra.run.smithi001.stdout:55834574852 2021-01-15T19:08:01.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2021-01-15T19:08:01.836 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2021-01-15T19:08:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:08:02.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2021-01-15T19:08:02.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2021-01-15T19:08:02.483 INFO:teuthology.orchestra.run.smithi001.stdout:55834574852 2021-01-15T19:08:02.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.4 2021-01-15T19:08:02.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2021-01-15T19:08:02.809 INFO:teuthology.orchestra.run.smithi001.stdout:55834574853 2021-01-15T19:08:02.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.5 2021-01-15T19:08:02.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2021-01-15T19:08:03.135 INFO:teuthology.orchestra.run.smithi001.stdout:55834574852 2021-01-15T19:08:03.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.6 2021-01-15T19:08:03.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2021-01-15T19:08:03.462 INFO:teuthology.orchestra.run.smithi001.stdout:55834574851 2021-01-15T19:08:03.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2021-01-15T19:08:03.476 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-01-15T19:08:03.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:03.785 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:03.787 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:03.797 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2021-01-15T19:08:03.507202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39872,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38400,"kb_avail":754934848,"statfs":{"total":773094113280,"available":773053284352,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39321600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.335029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.780249+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:07:56.780249+0000","last_peered":"2021-01-15T19:07:56.780249+0000","last_clean":"2021-01-15T19:07:56.780249+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:07:56.780249+0000","last_undegraded":"2021-01-15T19:07:56.780249+0000","last_fullsized":"2021-01-15T19:07:56.780249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.779861+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:07:56.779861+0000","last_peered":"2021-01-15T19:07:56.779861+0000","last_clean":"2021-01-15T19:07:56.779861+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:07:56.779861+0000","last_undegraded":"2021-01-15T19:07:56.779861+0000","last_fullsized":"2021-01-15T19:07:56.779861+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185950+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:07:56.185950+0000","last_peered":"2021-01-15T19:07:56.185950+0000","last_clean":"2021-01-15T19:07:56.185950+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:07:56.185950+0000","last_undegraded":"2021-01-15T19:07:56.185950+0000","last_fullsized":"2021-01-15T19:07:56.185950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185937+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:07:56.185937+0000","last_peered":"2021-01-15T19:07:56.185937+0000","last_clean":"2021-01-15T19:07:56.185937+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:07:56.185937+0000","last_undegraded":"2021-01-15T19:07:56.185937+0000","last_fullsized":"2021-01-15T19:07:56.185937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.176421+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:07:56.176421+0000","last_peered":"2021-01-15T19:07:56.176421+0000","last_clean":"2021-01-15T19:07:56.176421+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:07:56.176421+0000","last_undegraded":"2021-01-15T19:07:56.176421+0000","last_fullsized":"2021-01-15T19:07:56.176421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.183261+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:07:56.183261+0000","last_peered":"2021-01-15T19:07:56.183261+0000","last_clean":"2021-01-15T19:07:56.183261+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:07:56.183261+0000","last_undegraded":"2021-01-15T19:07:56.183261+0000","last_fullsized":"2021-01-15T19:07:56.183261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.186121+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:07:56.186121+0000","last_peered":"2021-01-15T19:07:56.186121+0000","last_clean":"2021-01-15T19:07:56.186121+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:07:56.186121+0000","last_undegraded":"2021-01-15T19:07:56.186121+0000","last_fullsized":"2021-01-15T19:07:56.186121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.946569+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:07:56.946569+0000","last_peered":"2021-01-15T19:07:56.946569+0000","last_clean":"2021-01-15T19:07:56.946569+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:07:56.946569+0000","last_undegraded":"2021-01-15T19:07:56.946569+0000","last_fullsized":"2021-01-15T19:07:56.946569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.947342+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:07:56.947342+0000","last_peered":"2021-01-15T19:07:56.947342+0000","last_clean":"2021-01-15T19:07:56.947342+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:07:56.947342+0000","last_undegraded":"2021-01-15T19:07:56.947342+0000","last_fullsized":"2021-01-15T19:07:56.947342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.764}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7310000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]}]}} 2021-01-15T19:08:03.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:04.107 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:04.119 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2021-01-15T19:08:03.507202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39872,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38400,"kb_avail":754934848,"statfs":{"total":773094113280,"available":773053284352,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39321600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.335029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.780249+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:07:56.780249+0000","last_peered":"2021-01-15T19:07:56.780249+0000","last_clean":"2021-01-15T19:07:56.780249+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:07:56.780249+0000","last_undegraded":"2021-01-15T19:07:56.780249+0000","last_fullsized":"2021-01-15T19:07:56.780249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.779861+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:07:56.779861+0000","last_peered":"2021-01-15T19:07:56.779861+0000","last_clean":"2021-01-15T19:07:56.779861+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:07:56.779861+0000","last_undegraded":"2021-01-15T19:07:56.779861+0000","last_fullsized":"2021-01-15T19:07:56.779861+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185950+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:07:56.185950+0000","last_peered":"2021-01-15T19:07:56.185950+0000","last_clean":"2021-01-15T19:07:56.185950+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:07:56.185950+0000","last_undegraded":"2021-01-15T19:07:56.185950+0000","last_fullsized":"2021-01-15T19:07:56.185950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185937+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:07:56.185937+0000","last_peered":"2021-01-15T19:07:56.185937+0000","last_clean":"2021-01-15T19:07:56.185937+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:07:56.185937+0000","last_undegraded":"2021-01-15T19:07:56.185937+0000","last_fullsized":"2021-01-15T19:07:56.185937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.176421+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:07:56.176421+0000","last_peered":"2021-01-15T19:07:56.176421+0000","last_clean":"2021-01-15T19:07:56.176421+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:07:56.176421+0000","last_undegraded":"2021-01-15T19:07:56.176421+0000","last_fullsized":"2021-01-15T19:07:56.176421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.183261+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:07:56.183261+0000","last_peered":"2021-01-15T19:07:56.183261+0000","last_clean":"2021-01-15T19:07:56.183261+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:07:56.183261+0000","last_undegraded":"2021-01-15T19:07:56.183261+0000","last_fullsized":"2021-01-15T19:07:56.183261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.186121+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:07:56.186121+0000","last_peered":"2021-01-15T19:07:56.186121+0000","last_clean":"2021-01-15T19:07:56.186121+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:07:56.186121+0000","last_undegraded":"2021-01-15T19:07:56.186121+0000","last_fullsized":"2021-01-15T19:07:56.186121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.946569+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:07:56.946569+0000","last_peered":"2021-01-15T19:07:56.946569+0000","last_clean":"2021-01-15T19:07:56.946569+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:07:56.946569+0000","last_undegraded":"2021-01-15T19:07:56.946569+0000","last_fullsized":"2021-01-15T19:07:56.946569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.947342+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:07:56.947342+0000","last_peered":"2021-01-15T19:07:56.947342+0000","last_clean":"2021-01-15T19:07:56.947342+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:07:56.947342+0000","last_undegraded":"2021-01-15T19:07:56.947342+0000","last_fullsized":"2021-01-15T19:07:56.947342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.764}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7310000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]}]}} 2021-01-15T19:08:04.121 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-01-15T19:08:04.122 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-01-15T19:08:04.122 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2021-01-15T19:08:04.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2021-01-15T19:08:04.500 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:04.513 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-01-15T19:08:04.513 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2021-01-15T19:08:04.514 INFO:teuthology.run_tasks:Running task thrashosds... 2021-01-15T19:08:04.526 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2021-01-15T19:08:04.526 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'master'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'master', 'sha1': '892c51dd3c5f7108e766bea30cd5e0d801b0abd3'}} 2021-01-15T19:08:04.527 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2021-01-15T19:08:04.527 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2021-01-15T19:08:04.860 INFO:tasks.thrashosds:Beginning thrashosds... 2021-01-15T19:08:04.861 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-01-15T19:08:04.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:05.171 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:05.172 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:05.182 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2021-01-15T19:08:03.507202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39872,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38400,"kb_avail":754934848,"statfs":{"total":773094113280,"available":773053284352,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39321600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.335029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.780249+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:07:56.780249+0000","last_peered":"2021-01-15T19:07:56.780249+0000","last_clean":"2021-01-15T19:07:56.780249+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:07:56.780249+0000","last_undegraded":"2021-01-15T19:07:56.780249+0000","last_fullsized":"2021-01-15T19:07:56.780249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.779861+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:07:56.779861+0000","last_peered":"2021-01-15T19:07:56.779861+0000","last_clean":"2021-01-15T19:07:56.779861+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:07:56.779861+0000","last_undegraded":"2021-01-15T19:07:56.779861+0000","last_fullsized":"2021-01-15T19:07:56.779861+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185950+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:07:56.185950+0000","last_peered":"2021-01-15T19:07:56.185950+0000","last_clean":"2021-01-15T19:07:56.185950+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:07:56.185950+0000","last_undegraded":"2021-01-15T19:07:56.185950+0000","last_fullsized":"2021-01-15T19:07:56.185950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185937+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:07:56.185937+0000","last_peered":"2021-01-15T19:07:56.185937+0000","last_clean":"2021-01-15T19:07:56.185937+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:07:56.185937+0000","last_undegraded":"2021-01-15T19:07:56.185937+0000","last_fullsized":"2021-01-15T19:07:56.185937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.176421+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:07:56.176421+0000","last_peered":"2021-01-15T19:07:56.176421+0000","last_clean":"2021-01-15T19:07:56.176421+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:07:56.176421+0000","last_undegraded":"2021-01-15T19:07:56.176421+0000","last_fullsized":"2021-01-15T19:07:56.176421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.183261+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:07:56.183261+0000","last_peered":"2021-01-15T19:07:56.183261+0000","last_clean":"2021-01-15T19:07:56.183261+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:07:56.183261+0000","last_undegraded":"2021-01-15T19:07:56.183261+0000","last_fullsized":"2021-01-15T19:07:56.183261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.186121+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:07:56.186121+0000","last_peered":"2021-01-15T19:07:56.186121+0000","last_clean":"2021-01-15T19:07:56.186121+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:07:56.186121+0000","last_undegraded":"2021-01-15T19:07:56.186121+0000","last_fullsized":"2021-01-15T19:07:56.186121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.946569+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:07:56.946569+0000","last_peered":"2021-01-15T19:07:56.946569+0000","last_clean":"2021-01-15T19:07:56.946569+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:07:56.946569+0000","last_undegraded":"2021-01-15T19:07:56.946569+0000","last_fullsized":"2021-01-15T19:07:56.946569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.947342+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:07:56.947342+0000","last_peered":"2021-01-15T19:07:56.947342+0000","last_clean":"2021-01-15T19:07:56.947342+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:07:56.947342+0000","last_undegraded":"2021-01-15T19:07:56.947342+0000","last_fullsized":"2021-01-15T19:07:56.947342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.764}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7310000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]}]}} 2021-01-15T19:08:05.184 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:05.495 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:05.497 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:05.508 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2021-01-15T19:08:03.507202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":39872,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":38400,"kb_avail":754934848,"statfs":{"total":773094113280,"available":773053284352,"internally_reserved":0,"allocated":1441792,"data_stored":476432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39321600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.335029"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.780249+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:07:56.780249+0000","last_peered":"2021-01-15T19:07:56.780249+0000","last_clean":"2021-01-15T19:07:56.780249+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:07:56.780249+0000","last_undegraded":"2021-01-15T19:07:56.780249+0000","last_fullsized":"2021-01-15T19:07:56.780249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.779861+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:07:56.779861+0000","last_peered":"2021-01-15T19:07:56.779861+0000","last_clean":"2021-01-15T19:07:56.779861+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:07:56.779861+0000","last_undegraded":"2021-01-15T19:07:56.779861+0000","last_fullsized":"2021-01-15T19:07:56.779861+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185950+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:07:56.185950+0000","last_peered":"2021-01-15T19:07:56.185950+0000","last_clean":"2021-01-15T19:07:56.185950+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:07:56.185950+0000","last_undegraded":"2021-01-15T19:07:56.185950+0000","last_fullsized":"2021-01-15T19:07:56.185950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.185937+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:07:56.185937+0000","last_peered":"2021-01-15T19:07:56.185937+0000","last_clean":"2021-01-15T19:07:56.185937+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:07:56.185937+0000","last_undegraded":"2021-01-15T19:07:56.185937+0000","last_fullsized":"2021-01-15T19:07:56.185937+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"14","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.176421+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:07:56.176421+0000","last_peered":"2021-01-15T19:07:56.176421+0000","last_clean":"2021-01-15T19:07:56.176421+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:07:56.176421+0000","last_undegraded":"2021-01-15T19:07:56.176421+0000","last_fullsized":"2021-01-15T19:07:56.176421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.183261+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:07:56.183261+0000","last_peered":"2021-01-15T19:07:56.183261+0000","last_clean":"2021-01-15T19:07:56.183261+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:07:56.183261+0000","last_undegraded":"2021-01-15T19:07:56.183261+0000","last_fullsized":"2021-01-15T19:07:56.183261+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.186121+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:07:56.186121+0000","last_peered":"2021-01-15T19:07:56.186121+0000","last_clean":"2021-01-15T19:07:56.186121+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:07:56.186121+0000","last_undegraded":"2021-01-15T19:07:56.186121+0000","last_fullsized":"2021-01-15T19:07:56.186121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.946569+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:07:56.946569+0000","last_peered":"2021-01-15T19:07:56.946569+0000","last_clean":"2021-01-15T19:07:56.946569+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:07:56.946569+0000","last_undegraded":"2021-01-15T19:07:56.946569+0000","last_fullsized":"2021-01-15T19:07:56.946569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"17","state":"active+clean","last_fresh":"2021-01-15T19:07:56.947342+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:07:56.947342+0000","last_peered":"2021-01-15T19:07:56.947342+0000","last_clean":"2021-01-15T19:07:56.947342+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:07:56.947342+0000","last_undegraded":"2021-01-15T19:07:56.947342+0000","last_fullsized":"2021-01-15T19:07:56.947342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.764}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.865}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7310000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4984,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":180224,"data_stored":59554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]}]}} 2021-01-15T19:08:05.509 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-01-15T19:08:05.510 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2021-01-15T19:08:05.823 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 18 2021-01-15T19:08:05.823 INFO:teuthology.orchestra.run.smithi001.stdout:fsid fe0e1890-043f-40bf-ad11-c5f08349f418 2021-01-15T19:08:05.823 INFO:teuthology.orchestra.run.smithi001.stdout:created 2021-01-15T19:07:43.378003+0000 2021-01-15T19:08:05.824 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2021-01-15T19:07:57.172007+0000 2021-01-15T19:08:05.824 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2021-01-15T19:08:05.824 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 4 2021-01-15T19:08:05.824 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2021-01-15T19:08:05.825 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2021-01-15T19:08:05.825 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2021-01-15T19:08:05.825 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2021-01-15T19:08:05.825 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client jewel 2021-01-15T19:08:05.825 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release pacific 2021-01-15T19:08:05.826 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2021-01-15T19:08:05.826 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 'device_health_metrics' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_min 1 application mgr_devicehealth 2021-01-15T19:08:05.826 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 application rbd 2021-01-15T19:08:05.826 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 8 2021-01-15T19:08:05.826 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6800/34833,v1:172.21.15.1:6801/34833] [v2:172.21.15.1:6802/34833,v1:172.21.15.1:6803/34833] exists,up e5b26f89-5c99-4764-9183-f3bb27c118f5 2021-01-15T19:08:05.827 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6808/34834,v1:172.21.15.1:6809/34834] [v2:172.21.15.1:6810/34834,v1:172.21.15.1:6811/34834] exists,up 99c0d455-1bd6-43aa-bfec-b6bddf87a35e 2021-01-15T19:08:05.827 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6817/34863,v1:172.21.15.1:6819/34863] [v2:172.21.15.1:6821/34863,v1:172.21.15.1:6823/34863] exists,up 71fe6c4b-3d73-4e07-b49d-98596ca7d9da 2021-01-15T19:08:05.827 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6816/34864,v1:172.21.15.1:6818/34864] [v2:172.21.15.1:6820/34864,v1:172.21.15.1:6822/34864] exists,up 8a6b018a-3a48-4a01-8526-a386f62ecb36 2021-01-15T19:08:05.828 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/34829,v1:172.21.15.136:6801/34829] [v2:172.21.15.136:6802/34829,v1:172.21.15.136:6803/34829] exists,up ff089622-e642-4bd0-901c-60efdb87a549 2021-01-15T19:08:05.828 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6809/34830,v1:172.21.15.136:6811/34830] [v2:172.21.15.136:6813/34830,v1:172.21.15.136:6815/34830] exists,up 3c57c228-dfaa-4e33-bded-27da0048d141 2021-01-15T19:08:05.828 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/34831,v1:172.21.15.136:6810/34831] [v2:172.21.15.136:6812/34831,v1:172.21.15.136:6814/34831] exists,up 201f889f-dac1-4a26-a285-7a0ec21c3646 2021-01-15T19:08:05.828 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/34874,v1:172.21.15.136:6825/34874] [v2:172.21.15.136:6826/34874,v1:172.21.15.136:6827/34874] exists,up 79ffd413-a9d3-4180-9691-d834d89c3f2e 2021-01-15T19:08:05.836 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6800/34833,v1:172.21.15.1:6801/34833] [v2:172.21.15.1:6802/34833,v1:172.21.15.1:6803/34833] exists,up e5b26f89-5c99-4764-9183-f3bb27c118f5', 'osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6808/34834,v1:172.21.15.1:6809/34834] [v2:172.21.15.1:6810/34834,v1:172.21.15.1:6811/34834] exists,up 99c0d455-1bd6-43aa-bfec-b6bddf87a35e', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6817/34863,v1:172.21.15.1:6819/34863] [v2:172.21.15.1:6821/34863,v1:172.21.15.1:6823/34863] exists,up 71fe6c4b-3d73-4e07-b49d-98596ca7d9da', 'osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6816/34864,v1:172.21.15.1:6818/34864] [v2:172.21.15.1:6820/34864,v1:172.21.15.1:6822/34864] exists,up 8a6b018a-3a48-4a01-8526-a386f62ecb36', 'osd.4 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/34829,v1:172.21.15.136:6801/34829] [v2:172.21.15.136:6802/34829,v1:172.21.15.136:6803/34829] exists,up ff089622-e642-4bd0-901c-60efdb87a549', 'osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6809/34830,v1:172.21.15.136:6811/34830] [v2:172.21.15.136:6813/34830,v1:172.21.15.136:6815/34830] exists,up 3c57c228-dfaa-4e33-bded-27da0048d141', 'osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/34831,v1:172.21.15.136:6810/34831] [v2:172.21.15.136:6812/34831,v1:172.21.15.136:6814/34831] exists,up 201f889f-dac1-4a26-a285-7a0ec21c3646', 'osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/34874,v1:172.21.15.136:6825/34874] [v2:172.21.15.136:6826/34874,v1:172.21.15.136:6827/34874] exists,up 79ffd413-a9d3-4180-9691-d834d89c3f2e'] 2021-01-15T19:08:05.836 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2021-01-15T19:08:05.963 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:05.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mon.a", 2021-01-15T19:08:05.964 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster": "ceph", 2021-01-15T19:08:05.964 INFO:teuthology.orchestra.run.smithi001.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2021-01-15T19:08:05.964 INFO:teuthology.orchestra.run.smithi001.stdout: "admin_socket_mode": "", 2021-01-15T19:08:05.965 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_client_required": "cephx, none", 2021-01-15T19:08:05.965 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_cluster_required": "cephx", 2021-01-15T19:08:05.965 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_debug": "true", 2021-01-15T19:08:05.965 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_mon_ticket_ttl": "43200.000000", 2021-01-15T19:08:05.966 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_service_required": "cephx", 2021-01-15T19:08:05.966 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_service_ticket_ttl": "3600.000000", 2021-01-15T19:08:05.966 INFO:teuthology.orchestra.run.smithi001.stdout: "auth_supported": "cephx", 2021-01-15T19:08:05.966 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio": "true", 2021-01-15T19:08:05.966 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_max_queue_depth": "1024", 2021-01-15T19:08:05.967 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_poll_ms": "250", 2021-01-15T19:08:05.967 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_aio_reap_max": "16", 2021-01-15T19:08:05.967 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_async_discard": "false", 2021-01-15T19:08:05.967 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_block_size": "4096", 2021-01-15T19:08:05.967 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio": "false", 2021-01-15T19:08:05.968 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio_log_age": "5.000000", 2021-01-15T19:08:05.968 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2021-01-15T19:08:05.968 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_debug_inflight_ios": "false", 2021-01-15T19:08:05.968 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_enable_discard": "false", 2021-01-15T19:08:05.969 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_flock_retry": "3", 2021-01-15T19:08:05.969 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_flock_retry_interval": "0.100000", 2021-01-15T19:08:05.969 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_inject_crash": "0", 2021-01-15T19:08:05.970 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_inject_crash_flush_delay": "2", 2021-01-15T19:08:05.970 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring": "false", 2021-01-15T19:08:05.970 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring_hipri": "false", 2021-01-15T19:08:05.970 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_ioring_sqthread_poll": "false", 2021-01-15T19:08:05.970 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_nvme_unbind_from_kernel": "false", 2021-01-15T19:08:05.971 INFO:teuthology.orchestra.run.smithi001.stdout: "bdev_type": "", 2021-01-15T19:08:05.971 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_alloc_size": "1048576", 2021-01-15T19:08:05.971 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_allocator": "hybrid", 2021-01-15T19:08:05.971 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_buffered_io": "false", 2021-01-15T19:08:05.972 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_compact_log_sync": "false", 2021-01-15T19:08:05.972 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2021-01-15T19:08:05.972 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_compact_min_size": "16777216", 2021-01-15T19:08:05.972 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_log_replay_check_allocations": "true", 2021-01-15T19:08:05.972 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_max_log_runway": "4194304", 2021-01-15T19:08:05.973 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_max_prefetch": "1048576", 2021-01-15T19:08:05.973 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_min_flush_size": "524288", 2021-01-15T19:08:05.973 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_min_log_runway": "1048576", 2021-01-15T19:08:05.973 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_replay_recovery": "false", 2021-01-15T19:08:05.973 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_replay_recovery_disable_compact": "false", 2021-01-15T19:08:05.974 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_shared_alloc_size": "65536", 2021-01-15T19:08:05.974 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_sync_write": "false", 2021-01-15T19:08:05.974 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2021-01-15T19:08:05.974 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2021-01-15T19:08:05.975 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2021-01-15T19:08:05.975 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_allocator": "hybrid", 2021-01-15T19:08:05.975 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2021-01-15T19:08:05.977 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2021-01-15T19:08:05.978 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2021-01-15T19:08:05.978 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bitmapallocator_span_size": "1024", 2021-01-15T19:08:05.978 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blobid_prealloc": "10240", 2021-01-15T19:08:05.978 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_create": "true", 2021-01-15T19:08:05.979 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_create": "false", 2021-01-15T19:08:05.979 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_path": "", 2021-01-15T19:08:05.979 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_db_size": "0", 2021-01-15T19:08:05.980 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_path": "", 2021-01-15T19:08:05.980 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_preallocate_file": "false", 2021-01-15T19:08:05.980 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_size": "107374182400", 2021-01-15T19:08:05.980 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_create": "false", 2021-01-15T19:08:05.981 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_path": "", 2021-01-15T19:08:05.981 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_block_wal_size": "100663296", 2021-01-15T19:08:05.981 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs": "true", 2021-01-15T19:08:05.981 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2021-01-15T19:08:05.982 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_bluefs_env_mirror": "false", 2021-01-15T19:08:05.982 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_autotune": "true", 2021-01-15T19:08:05.982 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_autotune_interval": "5.000000", 2021-01-15T19:08:05.982 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_kv_ratio": "0.400000", 2021-01-15T19:08:05.983 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_ratio": "0.400000", 2021-01-15T19:08:05.983 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size": "0", 2021-01-15T19:08:05.983 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size_hdd": "1073741824", 2021-01-15T19:08:05.983 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_size_ssd": "3221225472", 2021-01-15T19:08:05.984 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_trim_interval": "0.050000", 2021-01-15T19:08:05.984 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_trim_max_skip_pinned": "64", 2021-01-15T19:08:05.984 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_type": "2q", 2021-01-15T19:08:05.985 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_clone_cow": "true", 2021-01-15T19:08:05.985 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_algorithm": "snappy", 2021-01-15T19:08:05.986 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size": "0", 2021-01-15T19:08:05.986 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2021-01-15T19:08:05.986 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2021-01-15T19:08:05.986 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size": "0", 2021-01-15T19:08:05.987 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2021-01-15T19:08:05.987 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2021-01-15T19:08:05.987 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_mode": "none", 2021-01-15T19:08:05.988 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_compression_required_ratio": "0.875000", 2021-01-15T19:08:05.988 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_csum_type": "crc32c", 2021-01-15T19:08:05.988 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_enforce_settings": "default", 2021-01-15T19:08:05.988 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_freelist": "false", 2021-01-15T19:08:05.988 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_fsck_abort": "false", 2021-01-15T19:08:05.989 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_bug21040": "false", 2021-01-15T19:08:05.989 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2021-01-15T19:08:05.989 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_inject_read_err": "false", 2021-01-15T19:08:05.989 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_no_reuse_blocks": "false", 2021-01-15T19:08:05.990 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_omit_block_device_write": "false", 2021-01-15T19:08:05.990 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_omit_kv_commit": "false", 2021-01-15T19:08:05.990 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2021-01-15T19:08:05.990 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_prefill": "0.000000", 2021-01-15T19:08:05.991 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_prefragment_max": "1048576", 2021-01-15T19:08:05.991 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_random_read_err": "0.000000", 2021-01-15T19:08:05.991 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2021-01-15T19:08:05.991 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_small_allocations": "0", 2021-01-15T19:08:05.992 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2021-01-15T19:08:05.992 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_default_buffered_read": "true", 2021-01-15T19:08:05.992 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_default_buffered_write": "false", 2021-01-15T19:08:05.992 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops": "0", 2021-01-15T19:08:05.993 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2021-01-15T19:08:05.993 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2021-01-15T19:08:05.993 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2021-01-15T19:08:05.993 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_max_size": "1200", 2021-01-15T19:08:05.993 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_min_size": "150", 2021-01-15T19:08:05.994 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_target_size": "500", 2021-01-15T19:08:05.995 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2021-01-15T19:08:05.995 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_freelist_blocks_per_key": "128", 2021-01-15T19:08:05.995 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2021-01-15T19:08:05.996 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2021-01-15T19:08:05.996 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mkfs": "true", 2021-01-15T19:08:05.996 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2021-01-15T19:08:05.996 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mount": "false", 2021-01-15T19:08:05.997 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_mount_deep": "false", 2021-01-15T19:08:05.997 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_umount": "false", 2021-01-15T19:08:05.997 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_on_umount_deep": "false", 2021-01-15T19:08:05.997 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_quick_fix_on_mount": "true", 2021-01-15T19:08:05.998 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_quick_fix_threads": "2", 2021-01-15T19:08:05.998 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2021-01-15T19:08:05.998 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_gc_enable_blob_threshold": "0", 2021-01-15T19:08:05.998 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_gc_enable_total_threshold": "0", 2021-01-15T19:08:05.999 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2021-01-15T19:08:05.999 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_ignore_data_csum": "false", 2021-01-15T19:08:05.999 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2021-01-15T19:08:05.999 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_kvbackend": "rocksdb", 2021-01-15T19:08:06.000 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_collection_list_age": "60.000000", 2021-01-15T19:08:06.000 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2021-01-15T19:08:06.000 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_log_op_age": "5.000000", 2021-01-15T19:08:06.000 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_alloc_size": "0", 2021-01-15T19:08:06.000 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size": "0", 2021-01-15T19:08:06.001 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size_hdd": "65536", 2021-01-15T19:08:06.001 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_blob_size_ssd": "65536", 2021-01-15T19:08:06.001 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_defer_interval": "3.000000", 2021-01-15T19:08:06.002 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_max_deferred_txc": "32", 2021-01-15T19:08:06.002 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size": "0", 2021-01-15T19:08:06.002 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size_hdd": "4096", 2021-01-15T19:08:06.002 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_min_alloc_size_ssd": "4096", 2021-01-15T19:08:06.003 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_nid_prealloc": "1024", 2021-01-15T19:08:06.003 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size": "0", 2021-01-15T19:08:06.003 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2021-01-15T19:08:06.003 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2021-01-15T19:08:06.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_retry_disk_reads": "3", 2021-01-15T19:08:06.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_cf": "true", 2021-01-15T19:08:06.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13) L", 2021-01-15T19:08:06.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2021-01-15T19:08:06.004 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_rocksdb_options_annex": "", 2021-01-15T19:08:06.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_coremask": "0x1", 2021-01-15T19:08:06.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_io_sleep": "5", 2021-01-15T19:08:06.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_max_io_completion": "0", 2021-01-15T19:08:06.005 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_spdk_mem": "512", 2021-01-15T19:08:06.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_sync_submit_transaction": "false", 2021-01-15T19:08:06.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_bytes": "67108864", 2021-01-15T19:08:06.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io": "0", 2021-01-15T19:08:06.006 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2021-01-15T19:08:06.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2021-01-15T19:08:06.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2021-01-15T19:08:06.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_throttle_trace_rate": "0.000000", 2021-01-15T19:08:06.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_tracing": "false", 2021-01-15T19:08:06.007 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2021-01-15T19:08:06.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_reserved": "0", 2021-01-15T19:08:06.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2021-01-15T19:08:06.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2021-01-15T19:08:06.008 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_legacy_statfs": "true", 2021-01-15T19:08:06.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2021-01-15T19:08:06.009 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2021-01-15T19:08:06.009 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_path": "/usr/sbin/cephadm", 2021-01-15T19:08:06.009 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_action_update_interval": "2", 2021-01-15T19:08:06.010 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_directory_scan_interval": "10", 2021-01-15T19:08:06.010 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2021-01-15T19:08:06.010 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2021-01-15T19:08:06.011 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2021-01-15T19:08:06.011 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2021-01-15T19:08:06.011 INFO:teuthology.orchestra.run.smithi001.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2021-01-15T19:08:06.011 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_cluster_require_signatures": "false", 2021-01-15T19:08:06.012 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_cluster_require_version": "1", 2021-01-15T19:08:06.012 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_require_signatures": "false", 2021-01-15T19:08:06.012 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_require_version": "1", 2021-01-15T19:08:06.012 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_service_require_signatures": "false", 2021-01-15T19:08:06.013 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_service_require_version": "1", 2021-01-15T19:08:06.013 INFO:teuthology.orchestra.run.smithi001.stdout: "cephx_sign_messages": "true", 2021-01-15T19:08:06.013 INFO:teuthology.orchestra.run.smithi001.stdout: "chdir": "", 2021-01-15T19:08:06.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_acl_type": "", 2021-01-15T19:08:06.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_asio_thread_count": "2", 2021-01-15T19:08:06.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_cache_mid": "0.750000", 2021-01-15T19:08:06.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_cache_size": "16384", 2021-01-15T19:08:06.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_caps_release_delay": "5", 2021-01-15T19:08:06.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_check_pool_perm": "true", 2021-01-15T19:08:06.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_force_sync_read": "false", 2021-01-15T19:08:06.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_getattr_caps": "false", 2021-01-15T19:08:06.015 INFO:teuthology.orchestra.run.smithi001.stdout: "client_debug_inject_tick_delay": "0", 2021-01-15T19:08:06.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_die_on_failed_dentry_invalidate": "true", 2021-01-15T19:08:06.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_die_on_failed_remount": "false", 2021-01-15T19:08:06.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_dirsize_rbytes": "true", 2021-01-15T19:08:06.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_force_lazyio": "false", 2021-01-15T19:08:06.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_fs": "", 2021-01-15T19:08:06.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_inject_fixed_oldest_tid": "false", 2021-01-15T19:08:06.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_inject_release_failure": "false", 2021-01-15T19:08:06.017 INFO:teuthology.orchestra.run.smithi001.stdout: "client_max_inline_size": "4096", 2021-01-15T19:08:06.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mds_namespace": "", 2021-01-15T19:08:06.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_metadata": "", 2021-01-15T19:08:06.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_gid": "-1", 2021-01-15T19:08:06.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_timeout": "300.000000", 2021-01-15T19:08:06.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mount_uid": "-1", 2021-01-15T19:08:06.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_mountpoint": "/", 2021-01-15T19:08:06.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_notify_timeout": "10", 2021-01-15T19:08:06.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc": "true", 2021-01-15T19:08:06.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_dirty": "104857600", 2021-01-15T19:08:06.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_dirty_age": "5.000000", 2021-01-15T19:08:06.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_max_objects": "1000", 2021-01-15T19:08:06.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_size": "209715200", 2021-01-15T19:08:06.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_oc_target_dirty": "8388608", 2021-01-15T19:08:06.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_permissions": "true", 2021-01-15T19:08:06.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_quota_df": "true", 2021-01-15T19:08:06.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_max_bytes": "0", 2021-01-15T19:08:06.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_max_periods": "4", 2021-01-15T19:08:06.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_readahead_min": "131072", 2021-01-15T19:08:06.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_reconnect_stale": "false", 2021-01-15T19:08:06.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_shutdown_timeout": "30", 2021-01-15T19:08:06.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_snapdir": ".snap", 2021-01-15T19:08:06.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_tick_interval": "1", 2021-01-15T19:08:06.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_trace": "", 2021-01-15T19:08:06.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_try_dentry_invalidate": "false", 2021-01-15T19:08:06.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_use_faked_inos": "false", 2021-01-15T19:08:06.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_use_random_mds": "false", 2021-01-15T19:08:06.024 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog": "false", 2021-01-15T19:08:06.024 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog_host": "127.0.0.1", 2021-01-15T19:08:06.024 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_graylog_port": "12201", 2021-01-15T19:08:06.025 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_monitors": "default=true", 2021-01-15T19:08:06.025 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog": "false", 2021-01-15T19:08:06.025 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2021-01-15T19:08:06.025 INFO:teuthology.orchestra.run.smithi001.stdout: "clog_to_syslog_level": "info", 2021-01-15T19:08:06.026 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "-", 2021-01-15T19:08:06.026 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_network": "", 2021-01-15T19:08:06.026 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_network_interface": "", 2021-01-15T19:08:06.026 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_isal": "false", 2021-01-15T19:08:06.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_level": "5", 2021-01-15T19:08:06.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zlib_winsize": "-15", 2021-01-15T19:08:06.027 INFO:teuthology.orchestra.run.smithi001.stdout: "compressor_zstd_level": "1", 2021-01-15T19:08:06.027 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2021-01-15T19:08:06.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crash_dir": "/var/lib/ceph/crash", 2021-01-15T19:08:06.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_obc_lru_size": "10", 2021-01-15T19:08:06.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crimson_osd_scheduler_concurrency": "0", 2021-01-15T19:08:06.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "", 2021-01-15T19:08:06.028 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location_hook": "", 2021-01-15T19:08:06.029 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location_hook_timeout": "10", 2021-01-15T19:08:06.029 INFO:teuthology.orchestra.run.smithi001.stdout: "daemonize": "false", 2021-01-15T19:08:06.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_allow_any_pool_priority": "false", 2021-01-15T19:08:06.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asok": "1/5", 2021-01-15T19:08:06.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asok_assert_abort": "false", 2021-01-15T19:08:06.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_asserts_on_shutdown": "true", 2021-01-15T19:08:06.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_auth": "20/20", 2021-01-15T19:08:06.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bdev": "1/3", 2021-01-15T19:08:06.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bluefs": "1/5", 2021-01-15T19:08:06.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_bluestore": "1/5", 2021-01-15T19:08:06.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_buffer": "0/1", 2021-01-15T19:08:06.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_cephfs_mirror": "0/5", 2021-01-15T19:08:06.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_civetweb": "1/10", 2021-01-15T19:08:06.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_client": "0/5", 2021-01-15T19:08:06.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_compressor": "1/5", 2021-01-15T19:08:06.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_context": "0/1", 2021-01-15T19:08:06.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crush": "1/1", 2021-01-15T19:08:06.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_crypto": "1/5", 2021-01-15T19:08:06.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_deliberately_leak_memory": "false", 2021-01-15T19:08:06.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_disable_randomized_ping": "false", 2021-01-15T19:08:06.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_dpdk": "1/5", 2021-01-15T19:08:06.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_eventtrace": "1/5", 2021-01-15T19:08:06.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_filer": "0/1", 2021-01-15T19:08:06.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_filestore": "1/3", 2021-01-15T19:08:06.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_finisher": "1/1", 2021-01-15T19:08:06.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_fuse": "1/5", 2021-01-15T19:08:06.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_heartbeat_testing_span": "0", 2021-01-15T19:08:06.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_heartbeatmap": "1/5", 2021-01-15T19:08:06.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_immutable_obj_cache": "0/5", 2021-01-15T19:08:06.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_javaclient": "1/5", 2021-01-15T19:08:06.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_journal": "1/3", 2021-01-15T19:08:06.035 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_journaler": "0/5", 2021-01-15T19:08:06.036 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_kstore": "1/5", 2021-01-15T19:08:06.036 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_leveldb": "4/5", 2021-01-15T19:08:06.036 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_lockdep": "0/1", 2021-01-15T19:08:06.036 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds": "1/5", 2021-01-15T19:08:06.037 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_balancer": "1/5", 2021-01-15T19:08:06.037 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_locker": "1/5", 2021-01-15T19:08:06.037 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_log": "1/5", 2021-01-15T19:08:06.037 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_log_expire": "1/5", 2021-01-15T19:08:06.038 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mds_migrator": "1/5", 2021-01-15T19:08:06.038 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_memdb": "4/5", 2021-01-15T19:08:06.038 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mgr": "1/5", 2021-01-15T19:08:06.038 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mgrc": "1/5", 2021-01-15T19:08:06.039 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_mon": "20/20", 2021-01-15T19:08:06.039 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_monc": "0/10", 2021-01-15T19:08:06.039 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_ms": "1/1", 2021-01-15T19:08:06.039 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_none": "0/5", 2021-01-15T19:08:06.039 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objclass": "0/5", 2021-01-15T19:08:06.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objectcacher": "0/5", 2021-01-15T19:08:06.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_objecter": "0/1", 2021-01-15T19:08:06.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_optracker": "0/5", 2021-01-15T19:08:06.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_osd": "1/5", 2021-01-15T19:08:06.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_paxos": "20/20", 2021-01-15T19:08:06.041 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_perfcounter": "1/5", 2021-01-15T19:08:06.041 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_prioritycache": "1/5", 2021-01-15T19:08:06.041 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rados": "0/5", 2021-01-15T19:08:06.042 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd": "0/5", 2021-01-15T19:08:06.042 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_mirror": "0/5", 2021-01-15T19:08:06.042 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_pwl": "0/5", 2021-01-15T19:08:06.042 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rbd_replay": "0/5", 2021-01-15T19:08:06.042 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_refs": "0/0", 2021-01-15T19:08:06.043 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_reserver": "1/1", 2021-01-15T19:08:06.043 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw": "1/5", 2021-01-15T19:08:06.043 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rgw_sync": "1/5", 2021-01-15T19:08:06.043 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_rocksdb": "4/5", 2021-01-15T19:08:06.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_striper": "0/1", 2021-01-15T19:08:06.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_test": "0/5", 2021-01-15T19:08:06.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_throttle": "1/1", 2021-01-15T19:08:06.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_timer": "0/1", 2021-01-15T19:08:06.045 INFO:teuthology.orchestra.run.smithi001.stdout: "debug_tp": "0/5", 2021-01-15T19:08:06.045 INFO:teuthology.orchestra.run.smithi001.stdout: "device_failure_prediction_mode": "none", 2021-01-15T19:08:06.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2021-01-15T19:08:06.045 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2021-01-15T19:08:06.045 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_graylog": "false", 2021-01-15T19:08:06.046 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_stderr": "true", 2021-01-15T19:08:06.046 INFO:teuthology.orchestra.run.smithi001.stdout: "err_to_syslog": "false", 2021-01-15T19:08:06.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event_tracing": "false", 2021-01-15T19:08:06.046 INFO:teuthology.orchestra.run.smithi001.stdout: "fake_statfs_for_testing": "0", 2021-01-15T19:08:06.047 INFO:teuthology.orchestra.run.smithi001.stdout: "fatal_signal_handlers": "true", 2021-01-15T19:08:06.047 INFO:teuthology.orchestra.run.smithi001.stdout: "filer_max_purge_ops": "10", 2021-01-15T19:08:06.047 INFO:teuthology.orchestra.run.smithi001.stdout: "filer_max_truncate_ops": "128", 2021-01-15T19:08:06.047 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_apply_finisher_threads": "1", 2021-01-15T19:08:06.047 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_blackhole": "false", 2021-01-15T19:08:06.048 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_btrfs_clone_range": "true", 2021-01-15T19:08:06.048 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_btrfs_snap": "true", 2021-01-15T19:08:06.048 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_caller_concurrency": "10", 2021-01-15T19:08:06.048 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_collect_device_partition_information": "true", 2021-01-15T19:08:06.048 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_commit_timeout": "600.000000", 2021-01-15T19:08:06.049 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_inject_read_err": "false", 2021-01-15T19:08:06.049 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_omap_check": "false", 2021-01-15T19:08:06.049 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_random_read_err": "0.000000", 2021-01-15T19:08:06.049 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_debug_verify_split": "false", 2021-01-15T19:08:06.049 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_dump_file": "", 2021-01-15T19:08:06.050 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2021-01-15T19:08:06.050 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_expected_throughput_ops": "200.000000", 2021-01-15T19:08:06.050 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fadvise": "true", 2021-01-15T19:08:06.050 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fail_eio": "true", 2021-01-15T19:08:06.051 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fd_cache_shards": "16", 2021-01-15T19:08:06.051 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fd_cache_size": "128", 2021-01-15T19:08:06.051 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fiemap": "false", 2021-01-15T19:08:06.051 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fiemap_threshold": "4096", 2021-01-15T19:08:06.051 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_fsync_flushes_journal_data": "false", 2021-01-15T19:08:06.052 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_index_retry_probability": "0.000000", 2021-01-15T19:08:06.052 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_inject_stall": "0", 2021-01-15T19:08:06.052 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_parallel": "false", 2021-01-15T19:08:06.052 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_trailing": "false", 2021-01-15T19:08:06.052 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_journal_writeahead": "false", 2021-01-15T19:08:06.053 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_kill_at": "0", 2021-01-15T19:08:06.053 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_alloc_hint_size": "1048576", 2021-01-15T19:08:06.053 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size": "0", 2021-01-15T19:08:06.053 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2021-01-15T19:08:06.054 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_other": "512", 2021-01-15T19:08:06.054 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2021-01-15T19:08:06.054 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs": "0", 2021-01-15T19:08:06.055 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2021-01-15T19:08:06.055 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_other": "2", 2021-01-15T19:08:06.056 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_inline_xattrs_xfs": "10", 2021-01-15T19:08:06.056 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_sync_interval": "5.000000", 2021-01-15T19:08:06.056 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size": "0", 2021-01-15T19:08:06.056 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2021-01-15T19:08:06.057 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_other": "1024", 2021-01-15T19:08:06.057 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2021-01-15T19:08:06.057 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_merge_threshold": "-10", 2021-01-15T19:08:06.057 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_min_sync_interval": "0.010000", 2021-01-15T19:08:06.058 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_odsync_write": "false", 2021-01-15T19:08:06.058 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_backend": "rocksdb", 2021-01-15T19:08:06.058 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_backend_path": "", 2021-01-15T19:08:06.058 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_omap_header_cache_size": "1024", 2021-01-15T19:08:06.059 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_ondisk_finisher_threads": "1", 2021-01-15T19:08:06.059 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_thread_suicide_timeout": "180", 2021-01-15T19:08:06.059 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_thread_timeout": "60", 2021-01-15T19:08:06.059 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_op_threads": "2", 2021-01-15T19:08:06.060 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_punch_hole": "false", 2021-01-15T19:08:06.060 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2021-01-15T19:08:06.060 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2021-01-15T19:08:06.060 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2021-01-15T19:08:06.061 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_high_threshhold": "0.900000", 2021-01-15T19:08:06.061 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_low_threshhold": "0.300000", 2021-01-15T19:08:06.062 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_bytes": "104857600", 2021-01-15T19:08:06.062 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2021-01-15T19:08:06.062 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2021-01-15T19:08:06.063 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2021-01-15T19:08:06.063 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_queue_max_ops": "50", 2021-01-15T19:08:06.064 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2021-01-15T19:08:06.064 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_seek_data_hole": "false", 2021-01-15T19:08:06.064 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_sloppy_crc": "false", 2021-01-15T19:08:06.065 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_sloppy_crc_block_size": "65536", 2021-01-15T19:08:06.065 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_splice": "false", 2021-01-15T19:08:06.065 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_split_multiple": "2", 2021-01-15T19:08:06.066 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_split_rand_factor": "20", 2021-01-15T19:08:06.066 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_update_to": "1000", 2021-01-15T19:08:06.066 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2021-01-15T19:08:06.066 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2021-01-15T19:08:06.067 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2021-01-15T19:08:06.067 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2021-01-15T19:08:06.067 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2021-01-15T19:08:06.067 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2021-01-15T19:08:06.068 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_enable": "true", 2021-01-15T19:08:06.068 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2021-01-15T19:08:06.068 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2021-01-15T19:08:06.068 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2021-01-15T19:08:06.069 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2021-01-15T19:08:06.069 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2021-01-15T19:08:06.069 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2021-01-15T19:08:06.069 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_xfs_extsize": "false", 2021-01-15T19:08:06.070 INFO:teuthology.orchestra.run.smithi001.stdout: "filestore_zfs_snap": "false", 2021-01-15T19:08:06.070 INFO:teuthology.orchestra.run.smithi001.stdout: "fio_dir": "/tmp/fio", 2021-01-15T19:08:06.071 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "fe0e1890-043f-40bf-ad11-c5f08349f418", 2021-01-15T19:08:06.071 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_allow_other": "true", 2021-01-15T19:08:06.071 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_atomic_o_trunc": "true", 2021-01-15T19:08:06.072 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_big_writes": "true", 2021-01-15T19:08:06.072 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_debug": "false", 2021-01-15T19:08:06.073 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_default_permissions": "false", 2021-01-15T19:08:06.073 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_disable_pagecache": "false", 2021-01-15T19:08:06.073 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_max_write": "0", 2021-01-15T19:08:06.074 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_multithreaded": "true", 2021-01-15T19:08:06.074 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_require_active_mds": "true", 2021-01-15T19:08:06.075 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_set_user_groups": "true", 2021-01-15T19:08:06.075 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_move": "true", 2021-01-15T19:08:06.076 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_read": "true", 2021-01-15T19:08:06.076 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_splice_write": "true", 2021-01-15T19:08:06.077 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_syncfs_on_mksnap": "true", 2021-01-15T19:08:06.077 INFO:teuthology.orchestra.run.smithi001.stdout: "fuse_use_invalidate_cb": "true", 2021-01-15T19:08:06.077 INFO:teuthology.orchestra.run.smithi001.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2021-01-15T19:08:06.077 INFO:teuthology.orchestra.run.smithi001.stdout: "gss_target_name": "ceph", 2021-01-15T19:08:06.078 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_file": "", 2021-01-15T19:08:06.078 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_inject_failure": "0", 2021-01-15T19:08:06.079 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_interval": "5", 2021-01-15T19:08:06.079 INFO:teuthology.orchestra.run.smithi001.stdout: "host": "", 2021-01-15T19:08:06.080 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2021-01-15T19:08:06.080 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_max_inflight_ops": "128", 2021-01-15T19:08:06.080 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_max_size": "1073741824", 2021-01-15T19:08:06.081 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2021-01-15T19:08:06.081 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_burst": "0", 2021-01-15T19:08:06.081 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2021-01-15T19:08:06.082 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_bps_limit": "0", 2021-01-15T19:08:06.082 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_burst": "0", 2021-01-15T19:08:06.082 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2021-01-15T19:08:06.083 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_iops_limit": "0", 2021-01-15T19:08:06.083 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2021-01-15T19:08:06.083 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2021-01-15T19:08:06.084 INFO:teuthology.orchestra.run.smithi001.stdout: "immutable_object_cache_watermark": "0.900000", 2021-01-15T19:08:06.084 INFO:teuthology.orchestra.run.smithi001.stdout: "inject_early_sigterm": "false", 2021-01-15T19:08:06.084 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_aio": "true", 2021-01-15T19:08:06.084 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_align_min_size": "65536", 2021-01-15T19:08:06.085 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_block_align": "true", 2021-01-15T19:08:06.085 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_block_size": "4096", 2021-01-15T19:08:06.085 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_dio": "true", 2021-01-15T19:08:06.085 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_discard": "false", 2021-01-15T19:08:06.085 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_force_aio": "false", 2021-01-15T19:08:06.086 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_ignore_corruption": "false", 2021-01-15T19:08:06.086 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_max_write_bytes": "10485760", 2021-01-15T19:08:06.086 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_max_write_entries": "100", 2021-01-15T19:08:06.086 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_replay_from": "0", 2021-01-15T19:08:06.087 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_high_multiple": "0.000000", 2021-01-15T19:08:06.087 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_high_threshhold": "0.900000", 2021-01-15T19:08:06.087 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_low_threshhold": "0.600000", 2021-01-15T19:08:06.087 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_throttle_max_multiple": "0.000000", 2021-01-15T19:08:06.088 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_write_header_frequency": "0", 2021-01-15T19:08:06.088 INFO:teuthology.orchestra.run.smithi001.stdout: "journal_zero_on_create": "false", 2021-01-15T19:08:06.088 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_prefetch_periods": "10", 2021-01-15T19:08:06.088 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_prezero_periods": "5", 2021-01-15T19:08:06.089 INFO:teuthology.orchestra.run.smithi001.stdout: "journaler_write_head_interval": "15", 2021-01-15T19:08:06.089 INFO:teuthology.orchestra.run.smithi001.stdout: "key": "", 2021-01-15T19:08:06.089 INFO:teuthology.orchestra.run.smithi001.stdout: "keyfile": "", 2021-01-15T19:08:06.090 INFO:teuthology.orchestra.run.smithi001.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2021-01-15T19:08:06.090 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_backend": "rocksdb", 2021-01-15T19:08:06.090 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_default_stripe_size": "65536", 2021-01-15T19:08:06.090 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_fsck_on_mount": "false", 2021-01-15T19:08:06.090 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_fsck_on_mount_deep": "true", 2021-01-15T19:08:06.091 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_max_bytes": "67108864", 2021-01-15T19:08:06.091 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_max_ops": "512", 2021-01-15T19:08:06.091 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_nid_prealloc": "1024", 2021-01-15T19:08:06.091 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_onode_map_size": "1024", 2021-01-15T19:08:06.092 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2021-01-15T19:08:06.092 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_sync_submit_transaction": "false", 2021-01-15T19:08:06.092 INFO:teuthology.orchestra.run.smithi001.stdout: "kstore_sync_transaction": "false", 2021-01-15T19:08:06.092 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_block_size": "65536", 2021-01-15T19:08:06.092 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_bloom_size": "0", 2021-01-15T19:08:06.093 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_cache_size": "536870912", 2021-01-15T19:08:06.093 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_compact_on_mount": "false", 2021-01-15T19:08:06.093 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_compression": "false", 2021-01-15T19:08:06.093 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_log": "", 2021-01-15T19:08:06.093 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_log_to_ceph_log": "true", 2021-01-15T19:08:06.094 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_max_open_files": "0", 2021-01-15T19:08:06.094 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_paranoid": "false", 2021-01-15T19:08:06.094 INFO:teuthology.orchestra.run.smithi001.stdout: "leveldb_write_buffer_size": "33554432", 2021-01-15T19:08:06.094 INFO:teuthology.orchestra.run.smithi001.stdout: "librados_thread_count": "2", 2021-01-15T19:08:06.095 INFO:teuthology.orchestra.run.smithi001.stdout: "lockdep": "false", 2021-01-15T19:08:06.095 INFO:teuthology.orchestra.run.smithi001.stdout: "lockdep_force_backtrace": "false", 2021-01-15T19:08:06.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_coarse_timestamps": "true", 2021-01-15T19:08:06.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2021-01-15T19:08:06.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_flush_on_exit": "false", 2021-01-15T19:08:06.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_graylog_host": "127.0.0.1", 2021-01-15T19:08:06.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_graylog_port": "12201", 2021-01-15T19:08:06.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_max_new": "1000", 2021-01-15T19:08:06.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_max_recent": "10000", 2021-01-15T19:08:06.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_stderr_prefix": "", 2021-01-15T19:08:06.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_stop_at_utilization": "0.970000", 2021-01-15T19:08:06.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": "true", 2021-01-15T19:08:06.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_graylog": "false", 2021-01-15T19:08:06.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_stderr": "false", 2021-01-15T19:08:06.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_syslog": "false", 2021-01-15T19:08:06.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max_rotating_auth_attempts": "10", 2021-01-15T19:08:06.098 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_action_on_write_error": "1", 2021-01-15T19:08:06.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_asio_thread_count": "2", 2021-01-15T19:08:06.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_export_pin": "true", 2021-01-15T19:08:06.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_dirs": "true", 2021-01-15T19:08:06.099 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2021-01-15T19:08:06.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_interval": "5", 2021-01-15T19:08:06.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_fragment_size_max": "100000", 2021-01-15T19:08:06.100 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_idle_threshold": "0.000000", 2021-01-15T19:08:06.101 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_interval": "10", 2021-01-15T19:08:06.101 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_max": "-1", 2021-01-15T19:08:06.101 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_max_until": "-1", 2021-01-15T19:08:06.102 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_merge_size": "50", 2021-01-15T19:08:06.102 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_midchunk": "0.300000", 2021-01-15T19:08:06.102 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_min_rebalance": "0.100000", 2021-01-15T19:08:06.102 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_min_start": "0.200000", 2021-01-15T19:08:06.103 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_minchunk": "0.001000", 2021-01-15T19:08:06.103 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_mode": "0", 2021-01-15T19:08:06.103 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_need_max": "1.200000", 2021-01-15T19:08:06.103 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_need_min": "0.800000", 2021-01-15T19:08:06.104 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_replicate_threshold": "8000.000000", 2021-01-15T19:08:06.104 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_sample_interval": "3.000000", 2021-01-15T19:08:06.104 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_bits": "3", 2021-01-15T19:08:06.105 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_rd": "25000.000000", 2021-01-15T19:08:06.105 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_size": "10000", 2021-01-15T19:08:06.105 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_split_wr": "10000.000000", 2021-01-15T19:08:06.105 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_target_decay": "10.000000", 2021-01-15T19:08:06.105 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2021-01-15T19:08:06.106 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_grace": "15.000000", 2021-01-15T19:08:06.106 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_beacon_interval": "4.000000", 2021-01-15T19:08:06.106 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_memory_limit": "4294967296", 2021-01-15T19:08:06.106 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_mid": "0.700000", 2021-01-15T19:08:06.107 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_release_free_interval": "10", 2021-01-15T19:08:06.107 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_reservation": "0.050000", 2021-01-15T19:08:06.107 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_decay_rate": "1.000000", 2021-01-15T19:08:06.107 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_interval": "1", 2021-01-15T19:08:06.107 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache_trim_threshold": "262144", 2021-01-15T19:08:06.108 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2021-01-15T19:08:06.108 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2021-01-15T19:08:06.108 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_delegate_inos_pct": "50", 2021-01-15T19:08:06.108 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_prealloc_inos": "1000", 2021-01-15T19:08:06.108 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2021-01-15T19:08:06.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_damage_table_max_entries": "10000", 2021-01-15T19:08:06.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2021-01-15T19:08:06.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_auth_pins": "false", 2021-01-15T19:08:06.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_frag": "false", 2021-01-15T19:08:06.110 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_scatterstat": "false", 2021-01-15T19:08:06.110 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_debug_subtrees": "false", 2021-01-15T19:08:06.110 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_decay_halflife": "5.000000", 2021-01-15T19:08:06.110 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_default_dir_hash": "2", 2021-01-15T19:08:06.111 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_defer_session_stale": "true", 2021-01-15T19:08:06.111 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_deny_all_reconnect": "false", 2021-01-15T19:08:06.111 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_keys_per_op": "16384", 2021-01-15T19:08:06.111 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dir_max_commit_size": "10", 2021-01-15T19:08:06.111 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dirstat_min_interval": "1.000000", 2021-01-15T19:08:06.112 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_after_rejoin": "false", 2021-01-15T19:08:06.112 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_on_map": "false", 2021-01-15T19:08:06.112 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_threshold_file": "0", 2021-01-15T19:08:06.112 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2021-01-15T19:08:06.112 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_early_reply": "true", 2021-01-15T19:08:06.113 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_enable_op_tracker": "true", 2021-01-15T19:08:06.113 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_enforce_unique_name": "true", 2021-01-15T19:08:06.113 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_distributed": "true", 2021-01-15T19:08:06.113 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2021-01-15T19:08:06.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_random": "true", 2021-01-15T19:08:06.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_export_ephemeral_random_max": "0.010000", 2021-01-15T19:08:06.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_forward_all_requests_to_auth": "false", 2021-01-15T19:08:06.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_freeze_tree_timeout": "30.000000", 2021-01-15T19:08:06.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2021-01-15T19:08:06.115 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_health_cache_threshold": "1.500000", 2021-01-15T19:08:06.115 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_health_summarize_threshold": "10", 2021-01-15T19:08:06.115 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_heartbeat_grace": "15.000000", 2021-01-15T19:08:06.115 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_migrator_session_race": "false", 2021-01-15T19:08:06.116 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2021-01-15T19:08:06.116 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_join_fs": "", 2021-01-15T19:08:06.116 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_journal_format": "1", 2021-01-15T19:08:06.116 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_create_at": "0", 2021-01-15T19:08:06.116 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_export_at": "0", 2021-01-15T19:08:06.117 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_import_at": "0", 2021-01-15T19:08:06.117 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_at": "0", 2021-01-15T19:08:06.117 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_expire_at": "0", 2021-01-15T19:08:06.117 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_journal_replay_at": "0", 2021-01-15T19:08:06.117 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_link_at": "0", 2021-01-15T19:08:06.118 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_mdstable_at": "0", 2021-01-15T19:08:06.118 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_openc_at": "0", 2021-01-15T19:08:06.118 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_kill_rename_at": "0", 2021-01-15T19:08:06.118 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_events_per_segment": "1024", 2021-01-15T19:08:06.118 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_max_events": "-1", 2021-01-15T19:08:06.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_max_segments": "128", 2021-01-15T19:08:06.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_pause": "false", 2021-01-15T19:08:06.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_segment_size": "0", 2021-01-15T19:08:06.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_skip_corrupt_events": "false", 2021-01-15T19:08:06.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log_warn_factor": "2.000000", 2021-01-15T19:08:06.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_caps_per_client": "1048576", 2021-01-15T19:08:06.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_completed_flushes": "100000", 2021-01-15T19:08:06.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_completed_requests": "100000", 2021-01-15T19:08:06.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_export_size": "20971520", 2021-01-15T19:08:06.121 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_file_recover": "32", 2021-01-15T19:08:06.121 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_files": "64", 2021-01-15T19:08:06.121 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_ops": "8192", 2021-01-15T19:08:06.121 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2021-01-15T19:08:06.121 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_retries_on_remount_failure": "5", 2021-01-15T19:08:06.122 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_scrub_ops_in_progress": "5", 2021-01-15T19:08:06.122 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_snaps_per_dir": "100", 2021-01-15T19:08:06.122 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_max_xattr_pairs_size": "65536", 2021-01-15T19:08:06.122 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_metrics_update_interval": "2", 2021-01-15T19:08:06.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_min_caps_per_client": "100", 2021-01-15T19:08:06.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_min_caps_working_set": "10000", 2021-01-15T19:08:06.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mon_shutdown_timeout": "5.000000", 2021-01-15T19:08:06.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_numa_node": "-1", 2021-01-15T19:08:06.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_oft_prefetch_dirfrags": "true", 2021-01-15T19:08:06.124 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_complaint_time": "30.000000", 2021-01-15T19:08:06.124 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_duration": "600", 2021-01-15T19:08:06.124 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_history_size": "20", 2021-01-15T19:08:06.124 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_op_log_threshold": "5", 2021-01-15T19:08:06.124 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_ping_grace": "15", 2021-01-15T19:08:06.125 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_ping_interval": "5", 2021-01-15T19:08:06.125 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2021-01-15T19:08:06.125 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_global_max_decay_threshold": "131072", 2021-01-15T19:08:06.125 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_caps": "30000", 2021-01-15T19:08:06.125 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_decay_rate": "1.500000", 2021-01-15T19:08:06.126 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_max_decay_threshold": "131072", 2021-01-15T19:08:06.126 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_warning_decay_rate": "60.000000", 2021-01-15T19:08:06.126 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_recall_warning_threshold": "262144", 2021-01-15T19:08:06.126 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_reconnect_timeout": "45.000000", 2021-01-15T19:08:06.126 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_replay_interval": "1.000000", 2021-01-15T19:08:06.127 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_replay_unsafe_with_closed_session": "false", 2021-01-15T19:08:06.127 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_request_load_average_decay_rate": "60.000000", 2021-01-15T19:08:06.127 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_root_ino_gid": "0", 2021-01-15T19:08:06.127 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_root_ino_uid": "0", 2021-01-15T19:08:06.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_scatter_nudge_interval": "5.000000", 2021-01-15T19:08:06.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_blocklist_on_evict": "true", 2021-01-15T19:08:06.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_blocklist_on_timeout": "true", 2021-01-15T19:08:06.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2021-01-15T19:08:06.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cache_liveness_magnitude": "10", 2021-01-15T19:08:06.129 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2021-01-15T19:08:06.129 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_cap_acquisition_throttle": "500000", 2021-01-15T19:08:06.129 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2021-01-15T19:08:06.129 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessionmap_keys_per_op": "1024", 2021-01-15T19:08:06.129 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_shutdown_check": "0", 2021-01-15T19:08:06.130 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_skip_ino": "0", 2021-01-15T19:08:06.130 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_max_uid": "4294967294", 2021-01-15T19:08:06.130 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_min_uid": "0", 2021-01-15T19:08:06.130 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_snap_rstat": "false", 2021-01-15T19:08:06.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_task_status_update_interval": "2.000000", 2021-01-15T19:08:06.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_thrash_exports": "0", 2021-01-15T19:08:06.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_thrash_fragments": "0", 2021-01-15T19:08:06.131 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_tick_interval": "5.000000", 2021-01-15T19:08:06.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_verify_backtrace": "1", 2021-01-15T19:08:06.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_verify_scatter": "false", 2021-01-15T19:08:06.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_wipe_ino_prealloc": "false", 2021-01-15T19:08:06.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_wipe_sessions": "false", 2021-01-15T19:08:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool_debug": "false", 2021-01-15T19:08:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_debug_omit_block_device_write": "false", 2021-01-15T19:08:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_device_bytes": "1073741824", 2021-01-15T19:08:06.133 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_page_set": "false", 2021-01-15T19:08:06.134 INFO:teuthology.orchestra.run.smithi001.stdout: "memstore_page_size": "65536", 2021-01-15T19:08:06.134 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_bytes": "134217728", 2021-01-15T19:08:06.134 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_messages": "512", 2021-01-15T19:08:06.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2021-01-15T19:08:06.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_connect_retry_interval": "1.000000", 2021-01-15T19:08:06.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2021-01-15T19:08:06.135 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2021-01-15T19:08:06.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_disabled_modules": "", 2021-01-15T19:08:06.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_initial_modules": "restful iostat", 2021-01-15T19:08:06.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mds_bytes": "134217728", 2021-01-15T19:08:06.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mds_messages": "128", 2021-01-15T19:08:06.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2021-01-15T19:08:06.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mon_bytes": "134217728", 2021-01-15T19:08:06.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_mon_messages": "128", 2021-01-15T19:08:06.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_osd_bytes": "536870912", 2021-01-15T19:08:06.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_osd_messages": "8192", 2021-01-15T19:08:06.138 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_service_beacon_grace": "60.000000", 2021-01-15T19:08:06.138 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_stats_period": "5", 2021-01-15T19:08:06.138 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_stats_threshold": "5", 2021-01-15T19:08:06.138 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_tick_period": "2", 2021-01-15T19:08:06.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_accept_timeout_factor": "2.000000", 2021-01-15T19:08:06.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_allow_pool_delete": "true", 2021-01-15T19:08:06.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_allow_pool_size_one": "true", 2021-01-15T19:08:06.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_auth_validate_all_caps": "true", 2021-01-15T19:08:06.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2021-01-15T19:08:06.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2021-01-15T19:08:06.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_bytes": "104857600", 2021-01-15T19:08:06.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_directed_command_retry": "5", 2021-01-15T19:08:06.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval": "3.000000", 2021-01-15T19:08:06.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2021-01-15T19:08:06.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2021-01-15T19:08:06.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2021-01-15T19:08:06.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_hunt_parallel": "3", 2021-01-15T19:08:06.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_log_interval": "1.000000", 2021-01-15T19:08:06.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_max_log_entries_per_message": "1000", 2021-01-15T19:08:06.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_ping_interval": "10.000000", 2021-01-15T19:08:06.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_client_ping_timeout": "30.000000", 2021-01-15T19:08:06.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clock_drift_allowed": "1.000000", 2021-01-15T19:08:06.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2021-01-15T19:08:06.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2021-01-15T19:08:06.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_file_level": "debug", 2021-01-15T19:08:06.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_file": "true", 2021-01-15T19:08:06.145 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog": "false", 2021-01-15T19:08:06.145 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2021-01-15T19:08:06.145 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_graylog_port": "12201", 2021-01-15T19:08:06.146 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_stderr": "false", 2021-01-15T19:08:06.146 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog": "default=false", 2021-01-15T19:08:06.146 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2021-01-15T19:08:06.147 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cluster_log_to_syslog_level": "info", 2021-01-15T19:08:06.147 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_bootstrap": "false", 2021-01-15T19:08:06.147 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_start": "false", 2021-01-15T19:08:06.147 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_compact_on_trim": "true", 2021-01-15T19:08:06.148 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_con_tracker_persist_interval": "10", 2021-01-15T19:08:06.148 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_con_tracker_score_halflife": "43200", 2021-01-15T19:08:06.148 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_config_key_max_entry_size": "65536", 2021-01-15T19:08:06.148 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_cpu_threads": "4", 2021-01-15T19:08:06.149 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_crush_min_required_version": "hammer", 2021-01-15T19:08:06.149 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_daemon_bytes": "419430400", 2021-01-15T19:08:06.149 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2021-01-15T19:08:06.150 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_avail_crit": "5", 2021-01-15T19:08:06.150 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_avail_warn": "5", 2021-01-15T19:08:06.150 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_data_size_warn": "16106127360", 2021-01-15T19:08:06.150 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_block_osdmap_trim": "false", 2021-01-15T19:08:06.151 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_deprecated_as_obsolete": "false", 2021-01-15T19:08:06.151 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_json": "false", 2021-01-15T19:08:06.151 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2021-01-15T19:08:06.151 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_dump_transactions": "false", 2021-01-15T19:08:06.152 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_extra_checks": "false", 2021-01-15T19:08:06.152 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_initial_persistent_features": "false", 2021-01-15T19:08:06.152 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2021-01-15T19:08:06.152 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_octopus": "false", 2021-01-15T19:08:06.152 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_no_require_pacific": "false", 2021-01-15T19:08:06.153 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2021-01-15T19:08:06.153 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_delta_reset_interval": "10.000000", 2021-01-15T19:08:06.153 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_dns_srv_name": "ceph-mon", 2021-01-15T19:08:06.153 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_election_default_strategy": "1", 2021-01-15T19:08:06.154 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_election_timeout": "5.000000", 2021-01-15T19:08:06.154 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2021-01-15T19:08:06.154 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ping_divisor": "2", 2021-01-15T19:08:06.154 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_elector_ping_timeout": "2.000000", 2021-01-15T19:08:06.154 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_enable_op_tracker": "true", 2021-01-15T19:08:06.155 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_fake_pool_delete": "false", 2021-01-15T19:08:06.155 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_force_quorum_join": "false", 2021-01-15T19:08:06.155 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_globalid_prealloc": "10000", 2021-01-15T19:08:06.155 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_detail_to_clog": "false", 2021-01-15T19:08:06.156 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_log_update_period": "5", 2021-01-15T19:08:06.156 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_max_detail": "50", 2021-01-15T19:08:06.156 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog": "true", 2021-01-15T19:08:06.156 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog_interval": "600", 2021-01-15T19:08:06.156 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2021-01-15T19:08:06.157 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_host": "172.21.15.1,[v2:172.21.15.1:3301,v1:172.21.15.1:6790],172.21.15.136", 2021-01-15T19:08:06.157 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_host_override": "", 2021-01-15T19:08:06.157 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_initial_members": "", 2021-01-15T19:08:06.157 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2021-01-15T19:08:06.158 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2021-01-15T19:08:06.158 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_transaction_delay_max": "10.000000", 2021-01-15T19:08:06.158 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2021-01-15T19:08:06.158 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_keyvaluedb": "rocksdb", 2021-01-15T19:08:06.159 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease": "5.000000", 2021-01-15T19:08:06.159 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2021-01-15T19:08:06.159 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_lease_renew_interval_factor": "0.600000", 2021-01-15T19:08:06.159 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_log_max_summary": "50", 2021-01-15T19:08:06.160 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_log_entries_per_event": "4096", 2021-01-15T19:08:06.160 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_log_epochs": "500", 2021-01-15T19:08:06.160 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_mdsmap_epochs": "500", 2021-01-15T19:08:06.161 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_mgrmap_epochs": "500", 2021-01-15T19:08:06.161 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_osd": "10000", 2021-01-15T19:08:06.161 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_pg_per_osd": "10000", 2021-01-15T19:08:06.162 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_pool_pg_num": "65536", 2021-01-15T19:08:06.162 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_max_snap_prune_per_epoch": "100", 2021-01-15T19:08:06.162 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_blocklist_interval": "86400.000000", 2021-01-15T19:08:06.163 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_force_trim_to": "0", 2021-01-15T19:08:06.163 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mds_skip_sanity": "false", 2021-01-15T19:08:06.163 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_memory_autotune": "true", 2021-01-15T19:08:06.163 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_memory_target": "2147483648", 2021-01-15T19:08:06.164 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_beacon_grace": "30", 2021-01-15T19:08:06.164 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2021-01-15T19:08:06.164 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_digest_period": "5", 2021-01-15T19:08:06.164 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_inactive_grace": "60", 2021-01-15T19:08:06.164 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_mkfs_grace": "240", 2021-01-15T19:08:06.165 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2021-01-15T19:08:06.165 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_min_osdmap_epochs": "50", 2021-01-15T19:08:06.165 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_complaint_time": "30", 2021-01-15T19:08:06.165 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_duration": "600", 2021-01-15T19:08:06.166 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_size": "20", 2021-01-15T19:08:06.166 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_slow_op_size": "20", 2021-01-15T19:08:06.166 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_history_slow_op_threshold": "10", 2021-01-15T19:08:06.166 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_op_log_threshold": "5", 2021-01-15T19:08:06.167 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_adjust_down_out_interval": "true", 2021-01-15T19:08:06.167 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2021-01-15T19:08:06.167 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2021-01-15T19:08:06.167 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_in": "false", 2021-01-15T19:08:06.168 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_auto_mark_new_in": "true", 2021-01-15T19:08:06.168 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2021-01-15T19:08:06.168 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2021-01-15T19:08:06.168 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_cache_size": "500", 2021-01-15T19:08:06.169 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_cache_size_min": "134217728", 2021-01-15T19:08:06.169 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_crush_smoke_test": "true", 2021-01-15T19:08:06.169 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_destroyed_out_interval": "600", 2021-01-15T19:08:06.169 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_down_out_interval": "600", 2021-01-15T19:08:06.169 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_down_out_subtree_limit": "rack", 2021-01-15T19:08:06.170 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2021-01-15T19:08:06.170 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_force_trim_to": "0", 2021-01-15T19:08:06.170 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_full_ratio": "0.950000", 2021-01-15T19:08:06.170 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2021-01-15T19:08:06.171 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_halflife": "3600", 2021-01-15T19:08:06.171 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_max_interval": "300", 2021-01-15T19:08:06.171 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_laggy_weight": "0.300000", 2021-01-15T19:08:06.171 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2021-01-15T19:08:06.172 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_max_creating_pgs": "1024", 2021-01-15T19:08:06.172 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_max_initial_pgs": "1024", 2021-01-15T19:08:06.172 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_down_reporters": "2", 2021-01-15T19:08:06.173 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_in_ratio": "0.750000", 2021-01-15T19:08:06.173 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_min_up_ratio": "0.300000", 2021-01-15T19:08:06.173 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_nearfull_ratio": "0.850000", 2021-01-15T19:08:06.173 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp": "true", 2021-01-15T19:08:06.174 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2021-01-15T19:08:06.174 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2021-01-15T19:08:06.174 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_report_timeout": "900", 2021-01-15T19:08:06.175 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_reporter_subtree_level": "osd", 2021-01-15T19:08:06.175 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2021-01-15T19:08:06.175 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_warn_num_repaired": "10", 2021-01-15T19:08:06.176 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osd_warn_op_age": "32.000000", 2021-01-15T19:08:06.176 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_enabled": "true", 2021-01-15T19:08:06.176 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_interval": "2", 2021-01-15T19:08:06.176 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_min": "15", 2021-01-15T19:08:06.177 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_osdmap_full_prune_txsize": "2", 2021-01-15T19:08:06.177 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2021-01-15T19:08:06.177 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_stuck_threshold": "60", 2021-01-15T19:08:06.178 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2021-01-15T19:08:06.178 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_objects": "10000", 2021-01-15T19:08:06.178 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_per_osd": "0", 2021-01-15T19:08:06.179 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pg_warn_min_pool_objects": "1000", 2021-01-15T19:08:06.179 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pool_quota_crit_threshold": "0", 2021-01-15T19:08:06.179 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_pool_quota_warn_threshold": "0", 2021-01-15T19:08:06.179 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_probe_timeout": "2.000000", 2021-01-15T19:08:06.180 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_max_change": "0.050000", 2021-01-15T19:08:06.180 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_max_osds": "4", 2021-01-15T19:08:06.180 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_min_bytes_per_osd": "10", 2021-01-15T19:08:06.180 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_reweight_min_pgs_per_osd": "4", 2021-01-15T19:08:06.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2021-01-15T19:08:06.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2021-01-15T19:08:06.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2021-01-15T19:08:06.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_interval": "300", 2021-01-15T19:08:06.181 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_max_keys": "100", 2021-01-15T19:08:06.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_scrub_timeout": "300", 2021-01-15T19:08:06.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_session_timeout": "300", 2021-01-15T19:08:06.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_smart_report_timeout": "5", 2021-01-15T19:08:06.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stat_smooth_intervals": "6", 2021-01-15T19:08:06.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2021-01-15T19:08:06.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_pool_min_size": "2", 2021-01-15T19:08:06.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_pool_size": "4", 2021-01-15T19:08:06.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2021-01-15T19:08:06.183 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_subscribe_interval": "86400.000000", 2021-01-15T19:08:06.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_debug": "false", 2021-01-15T19:08:06.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_max_payload_keys": "2000", 2021-01-15T19:08:06.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_max_payload_size": "1048576", 2021-01-15T19:08:06.184 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_provider_kill_at": "0", 2021-01-15T19:08:06.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_requester_kill_at": "0", 2021-01-15T19:08:06.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_sync_timeout": "60.000000", 2021-01-15T19:08:06.185 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_target_pg_per_osd": "100", 2021-01-15T19:08:06.186 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_tick_interval": "5", 2021-01-15T19:08:06.186 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_timecheck_interval": "300.000000", 2021-01-15T19:08:06.186 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_timecheck_skew_interval": "30.000000", 2021-01-15T19:08:06.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_older_version_delay": "604800", 2021-01-15T19:08:06.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2021-01-15T19:08:06.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2021-01-15T19:08:06.187 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2021-01-15T19:08:06.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2021-01-15T19:08:06.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_misplaced": "false", 2021-01-15T19:08:06.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2021-01-15T19:08:06.189 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_older_version": "true", 2021-01-15T19:08:06.189 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2021-01-15T19:08:06.189 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_no_app": "true", 2021-01-15T19:08:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_no_redundancy": "false", 2021-01-15T19:08:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2021-01-15T19:08:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2021-01-15T19:08:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2021-01-15T19:08:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_on_too_few_osds": "false", 2021-01-15T19:08:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2021-01-15T19:08:06.192 INFO:teuthology.orchestra.run.smithi001.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2021-01-15T19:08:06.192 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": "", 2021-01-15T19:08:06.192 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_max_op_threads": "5", 2021-01-15T19:08:06.192 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_op_threads": "3", 2021-01-15T19:08:06.193 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_buffer_size": "131072", 2021-01-15T19:08:06.193 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_cm": "false", 2021-01-15T19:08:06.193 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_device_name": "", 2021-01-15T19:08:06.194 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_dscp": "96", 2021-01-15T19:08:06.194 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_enable_hugepage": "false", 2021-01-15T19:08:06.194 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_gid_idx": "0", 2021-01-15T19:08:06.195 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_local_gid": "", 2021-01-15T19:08:06.195 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_polling_us": "1000", 2021-01-15T19:08:06.195 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_port_num": "1", 2021-01-15T19:08:06.196 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_receive_buffers": "32768", 2021-01-15T19:08:06.196 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_receive_queue_len": "4096", 2021-01-15T19:08:06.196 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_roce_ver": "1", 2021-01-15T19:08:06.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_send_buffers": "1024", 2021-01-15T19:08:06.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_sl": "3", 2021-01-15T19:08:06.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_support_srq": "true", 2021-01-15T19:08:06.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_async_rdma_type": "ib", 2021-01-15T19:08:06.198 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_before_connect": "false", 2021-01-15T19:08:06.198 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_ipv4": "true", 2021-01-15T19:08:06.198 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_ipv6": "false", 2021-01-15T19:08:06.198 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_msgr1": "true", 2021-01-15T19:08:06.199 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_msgr2": "true", 2021-01-15T19:08:06.199 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_port_max": "7300", 2021-01-15T19:08:06.199 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_port_min": "6800", 2021-01-15T19:08:06.200 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_prefer_ipv4": "false", 2021-01-15T19:08:06.200 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_retry_count": "3", 2021-01-15T19:08:06.200 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_bind_retry_delay": "5", 2021-01-15T19:08:06.200 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_client": "false", 2021-01-15T19:08:06.201 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mds": "false", 2021-01-15T19:08:06.201 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mgr": "false", 2021-01-15T19:08:06.201 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_mon": "false", 2021-01-15T19:08:06.201 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_blackhole_osd": "false", 2021-01-15T19:08:06.202 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_client_mode": "crc secure", 2021-01-15T19:08:06.202 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_cluster_mode": "crc secure", 2021-01-15T19:08:06.202 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_cluster_type": "", 2021-01-15T19:08:06.202 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_connection_idle_timeout": "900", 2021-01-15T19:08:06.202 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_connection_ready_timeout": "10", 2021-01-15T19:08:06.203 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_crc_data": "true", 2021-01-15T19:08:06.203 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_crc_header": "true", 2021-01-15T19:08:06.203 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_bad_msg": "false", 2021-01-15T19:08:06.203 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_bug": "true", 2021-01-15T19:08:06.204 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_old_message": "true", 2021-01-15T19:08:06.204 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_skipped_message": "false", 2021-01-15T19:08:06.204 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_die_on_unhandled_msg": "false", 2021-01-15T19:08:06.204 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dispatch_throttle_bytes": "104857600", 2021-01-15T19:08:06.205 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_coremask": "0xF", 2021-01-15T19:08:06.205 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_debug_allow_loopback": "false", 2021-01-15T19:08:06.205 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2021-01-15T19:08:06.205 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_host_ipv4_addr": "", 2021-01-15T19:08:06.206 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hugepages": "", 2021-01-15T19:08:06.206 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hw_flow_control": "true", 2021-01-15T19:08:06.206 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2021-01-15T19:08:06.206 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_lro": "true", 2021-01-15T19:08:06.207 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_memory_channel": "4", 2021-01-15T19:08:06.207 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2021-01-15T19:08:06.207 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_pmd": "", 2021-01-15T19:08:06.208 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_port_id": "0", 2021-01-15T19:08:06.208 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2021-01-15T19:08:06.208 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dump_corrupt_message_level": "1", 2021-01-15T19:08:06.208 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_dump_on_send": "false", 2021-01-15T19:08:06.209 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_initial_backoff": "0.200000", 2021-01-15T19:08:06.209 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_max": "1.000000", 2021-01-15T19:08:06.209 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_probability": "0.000000", 2021-01-15T19:08:06.209 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_delay_type": "", 2021-01-15T19:08:06.209 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_internal_delays": "0.000000", 2021-01-15T19:08:06.214 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_inject_socket_failures": "5000", 2021-01-15T19:08:06.214 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_learn_addr_from_peer": "true", 2021-01-15T19:08:06.214 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_max_accept_failures": "4", 2021-01-15T19:08:06.215 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_max_backoff": "15.000000", 2021-01-15T19:08:06.215 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_client_mode": "secure crc", 2021-01-15T19:08:06.215 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_cluster_mode": "secure crc", 2021-01-15T19:08:06.215 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_mon_service_mode": "secure crc", 2021-01-15T19:08:06.216 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2021-01-15T19:08:06.216 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_pq_min_cost": "65536", 2021-01-15T19:08:06.216 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_public_type": "", 2021-01-15T19:08:06.216 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_service_mode": "crc secure", 2021-01-15T19:08:06.217 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_listen_backlog": "512", 2021-01-15T19:08:06.217 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_nodelay": "true", 2021-01-15T19:08:06.217 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_prefetch_max_size": "4096", 2021-01-15T19:08:06.217 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_tcp_rcvbuf": "0", 2021-01-15T19:08:06.218 INFO:teuthology.orchestra.run.smithi001.stdout: "ms_type": "async+posix", 2021-01-15T19:08:06.218 INFO:teuthology.orchestra.run.smithi001.stdout: "no_config_file": "false", 2021-01-15T19:08:06.218 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_completion_locks_per_session": "32", 2021-01-15T19:08:06.218 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_debug_inject_relock_delay": "false", 2021-01-15T19:08:06.219 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inflight_op_bytes": "104857600", 2021-01-15T19:08:06.219 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inflight_ops": "1024", 2021-01-15T19:08:06.219 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_inject_no_watch_ping": "false", 2021-01-15T19:08:06.220 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_retry_writes_after_first_reply": "false", 2021-01-15T19:08:06.220 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_tick_interval": "5.000000", 2021-01-15T19:08:06.220 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter_timeout": "10.000000", 2021-01-15T19:08:06.220 INFO:teuthology.orchestra.run.smithi001.stdout: "objectstore_blackhole": "false", 2021-01-15T19:08:06.221 INFO:teuthology.orchestra.run.smithi001.stdout: "openssl_engine_opts": "", 2021-01-15T19:08:06.221 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_delay_time": "5.000000", 2021-01-15T19:08:06.221 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_hist_halflife": "1000", 2021-01-15T19:08:06.221 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_max_low_ops": "2", 2021-01-15T19:08:06.222 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_max_ops": "4", 2021-01-15T19:08:06.222 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_min_evict_effort": "0.100000", 2021-01-15T19:08:06.222 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_quantize_effort": "0.100000", 2021-01-15T19:08:06.222 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_agent_slop": "0.020000", 2021-01-15T19:08:06.222 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_allow_recovery_below_min_size": "true", 2021-01-15T19:08:06.223 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_asio_thread_count": "2", 2021-01-15T19:08:06.223 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_async_recovery_min_cost": "1", 2021-01-15T19:08:06.223 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_auto_mark_unfound_lost": "false", 2021-01-15T19:08:06.223 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_retry_interval": "30.000000", 2021-01-15T19:08:06.224 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_scan_max": "512", 2021-01-15T19:08:06.224 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backfill_scan_min": "64", 2021-01-15T19:08:06.224 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_degraded": "false", 2021-01-15T19:08:06.224 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_peering": "false", 2021-01-15T19:08:06.224 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_backoff_on_unfound": "true", 2021-01-15T19:08:06.225 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_beacon_report_interval": "300", 2021-01-15T19:08:06.225 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_duration": "30", 2021-01-15T19:08:06.225 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_large_size_max_throughput": "104857600", 2021-01-15T19:08:06.225 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_max_block_size": "67108864", 2021-01-15T19:08:06.225 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bench_small_size_max_iops": "100", 2021-01-15T19:08:06.226 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_blkin_trace_all": "false", 2021-01-15T19:08:06.226 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2021-01-15T19:08:06.226 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2021-01-15T19:08:06.226 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_check_for_log_corruption": "false", 2021-01-15T19:08:06.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_check_max_object_name_len_on_startup": "true", 2021-01-15T19:08:06.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2021-01-15T19:08:06.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2021-01-15T19:08:06.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2021-01-15T19:08:06.227 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_class_update_on_start": "true", 2021-01-15T19:08:06.228 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_message_cap": "0", 2021-01-15T19:08:06.228 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_message_size_cap": "524288000", 2021-01-15T19:08:06.228 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_op_priority": "63", 2021-01-15T19:08:06.228 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_client_watch_timeout": "30", 2021-01-15T19:08:06.228 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_command_max_records": "256", 2021-01-15T19:08:06.229 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_command_thread_suicide_timeout": "900", 2021-01-15T19:08:06.229 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_command_thread_timeout": "600", 2021-01-15T19:08:06.229 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_compact_on_start": "false", 2021-01-15T19:08:06.230 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_copyfrom_max_chunk": "8388608", 2021-01-15T19:08:06.230 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_chooseleaf_type": "0", 2021-01-15T19:08:06.230 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_initial_weight": "-1.000000", 2021-01-15T19:08:06.230 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_update_on_start": "true", 2021-01-15T19:08:06.231 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_crush_update_weight_set": "true", 2021-01-15T19:08:06.231 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2021-01-15T19:08:06.231 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2021-01-15T19:08:06.231 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2021-01-15T19:08:06.231 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_drop_ping_duration": "0", 2021-01-15T19:08:06.232 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_drop_ping_probability": "0.000000", 2021-01-15T19:08:06.232 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_feed_pullee": "-1", 2021-01-15T19:08:06.232 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_copyfrom_error": "false", 2021-01-15T19:08:06.232 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2021-01-15T19:08:06.233 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2021-01-15T19:08:06.233 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_misdirected_ops": "false", 2021-01-15T19:08:06.233 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_no_acting_change": "false", 2021-01-15T19:08:06.233 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_no_purge_strays": "false", 2021-01-15T19:08:06.234 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_op_order": "false", 2021-01-15T19:08:06.234 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_pg_log_writeout": "false", 2021-01-15T19:08:06.234 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_pretend_recovery_active": "false", 2021-01-15T19:08:06.234 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_random_push_read_error": "0.000000", 2021-01-15T19:08:06.234 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2021-01-15T19:08:06.235 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_shutdown": "false", 2021-01-15T19:08:06.235 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2021-01-15T19:08:06.235 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2021-01-15T19:08:06.236 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_cached_snaps": "false", 2021-01-15T19:08:06.236 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_missing_on_start": "false", 2021-01-15T19:08:06.236 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_snaps": "false", 2021-01-15T19:08:06.236 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_debug_verify_stray_on_activate": "false", 2021-01-15T19:08:06.236 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_interval": "604800.000000", 2021-01-15T19:08:06.237 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_keys": "1024", 2021-01-15T19:08:06.237 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2021-01-15T19:08:06.237 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2021-01-15T19:08:06.237 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2021-01-15T19:08:06.237 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_stride": "524288", 2021-01-15T19:08:06.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2021-01-15T19:08:06.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_default_data_pool_replay_window": "5", 2021-01-15T19:08:06.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_default_notify_timeout": "30", 2021-01-15T19:08:06.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep": "0.000000", 2021-01-15T19:08:06.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_hdd": "5.000000", 2021-01-15T19:08:06.239 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_hybrid": "1.000000", 2021-01-15T19:08:06.239 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_delete_sleep_ssd": "1.000000", 2021-01-15T19:08:06.239 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_discard_disconnected_ops": "true", 2021-01-15T19:08:06.240 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_enable_op_tracker": "true", 2021-01-15T19:08:06.240 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2021-01-15T19:08:06.240 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_failsafe_full_ratio": "0.970000", 2021-01-15T19:08:06.240 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_fail_on_connection_refused": "true", 2021-01-15T19:08:06.240 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_info": "true", 2021-01-15T19:08:06.241 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_fast_shutdown": "true", 2021-01-15T19:08:06.241 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_find_best_info_ignore_history_les": "false", 2021-01-15T19:08:06.241 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_force_auth_primary_missing_objects": "100", 2021-01-15T19:08:06.241 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2021-01-15T19:08:06.241 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_function_tracing": "false", 2021-01-15T19:08:06.242 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_grace": "20", 2021-01-15T19:08:06.242 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_interval": "6", 2021-01-15T19:08:06.242 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2021-01-15T19:08:06.243 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_peers": "10", 2021-01-15T19:08:06.243 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_min_size": "2000", 2021-01-15T19:08:06.243 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_stale": "600", 2021-01-15T19:08:06.243 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2021-01-15T19:08:06.244 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_max_size": "100000", 2021-01-15T19:08:06.244 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_min_size": "1000", 2021-01-15T19:08:06.244 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_hit_set_namespace": ".ceph-internal", 2021-01-15T19:08:06.244 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_ignore_stale_divergent_priors": "false", 2021-01-15T19:08:06.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2021-01-15T19:08:06.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_inject_failure_on_pg_removal": "false", 2021-01-15T19:08:06.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2021-01-15T19:08:06.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal_flush_on_shutdown": "true", 2021-01-15T19:08:06.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_journal_size": "5120", 2021-01-15T19:08:06.246 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_kill_backfill_at": "0", 2021-01-15T19:08:06.246 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_loop_before_reset_tphandle": "64", 2021-01-15T19:08:06.246 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_cache_size": "50", 2021-01-15T19:08:06.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_dedup": "true", 2021-01-15T19:08:06.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_message_max": "40", 2021-01-15T19:08:06.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_message_max_bytes": "10485760", 2021-01-15T19:08:06.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_map_share_max_epochs": "40", 2021-01-15T19:08:06.247 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_attr_name_len": "100", 2021-01-15T19:08:06.248 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_attr_size": "0", 2021-01-15T19:08:06.248 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_backfills": "1", 2021-01-15T19:08:06.248 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_markdown_count": "5", 2021-01-15T19:08:06.248 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_markdown_period": "600", 2021-01-15T19:08:06.248 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_name_len": "2048", 2021-01-15T19:08:06.249 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_namespace_len": "256", 2021-01-15T19:08:06.249 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_object_size": "134217728", 2021-01-15T19:08:06.249 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2021-01-15T19:08:06.249 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_omap_entries_per_request": "1024", 2021-01-15T19:08:06.250 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_blocked_by": "16", 2021-01-15T19:08:06.250 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_log_entries": "10000", 2021-01-15T19:08:06.250 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2021-01-15T19:08:06.251 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_pgls": "1024", 2021-01-15T19:08:06.251 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_push_cost": "8388608", 2021-01-15T19:08:06.251 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_push_objects": "10", 2021-01-15T19:08:06.251 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_scrubs": "1", 2021-01-15T19:08:06.251 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2021-01-15T19:08:06.252 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_trimming_pgs": "2", 2021-01-15T19:08:06.252 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_write_op_reply_len": "32", 2021-01-15T19:08:06.252 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_max_write_size": "90", 2021-01-15T19:08:06.252 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2021-01-15T19:08:06.252 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2021-01-15T19:08:06.253 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2021-01-15T19:08:06.253 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2021-01-15T19:08:06.253 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2021-01-15T19:08:06.253 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2021-01-15T19:08:06.254 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2021-01-15T19:08:06.254 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_lim": "999999", 2021-01-15T19:08:06.254 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_res": "1", 2021-01-15T19:08:06.254 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mclock_scheduler_client_wgt": "1", 2021-01-15T19:08:06.254 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_base": "805306368", 2021-01-15T19:08:06.255 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_cache_min": "134217728", 2021-01-15T19:08:06.255 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_cache_resize_interval": "1.000000", 2021-01-15T19:08:06.255 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_expected_fragmentation": "0.150000", 2021-01-15T19:08:06.255 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target": "4294967296", 2021-01-15T19:08:06.256 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2021-01-15T19:08:06.256 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_min_pg_log_entries": "250", 2021-01-15T19:08:06.256 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_min_recovery_priority": "0", 2021-01-15T19:08:06.256 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_heartbeat_interval": "30", 2021-01-15T19:08:06.256 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2021-01-15T19:08:06.257 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_report_interval": "5", 2021-01-15T19:08:06.257 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_report_max_in_flight": "2", 2021-01-15T19:08:06.257 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mon_shutdown_timeout": "5.000000", 2021-01-15T19:08:06.257 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_num_cache_shards": "32", 2021-01-15T19:08:06.258 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_num_op_tracker_shard": "32", 2021-01-15T19:08:06.258 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_auto_affinity": "true", 2021-01-15T19:08:06.258 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_node": "-1", 2021-01-15T19:08:06.258 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_numa_prefer_iface": "true", 2021-01-15T19:08:06.258 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_object_clean_region_max_num_intervals": "10", 2021-01-15T19:08:06.259 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objecter_finishers": "1", 2021-01-15T19:08:06.259 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore": "bluestore", 2021-01-15T19:08:06.259 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore_fuse": "false", 2021-01-15T19:08:06.259 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_objectstore_tracing": "false", 2021-01-15T19:08:06.259 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_complaint_time": "30.000000", 2021-01-15T19:08:06.260 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_duration": "600", 2021-01-15T19:08:06.260 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_size": "20", 2021-01-15T19:08:06.260 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_slow_op_size": "20", 2021-01-15T19:08:06.260 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2021-01-15T19:08:06.260 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_log_threshold": "5", 2021-01-15T19:08:06.261 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards": "0", 2021-01-15T19:08:06.261 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards_hdd": "5", 2021-01-15T19:08:06.261 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_shards_ssd": "8", 2021-01-15T19:08:06.261 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard": "0", 2021-01-15T19:08:06.262 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2021-01-15T19:08:06.262 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2021-01-15T19:08:06.262 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2021-01-15T19:08:06.262 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_pq_min_cost": "65536", 2021-01-15T19:08:06.262 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_queue": "wpq", 2021-01-15T19:08:06.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_queue_cut_off": "high", 2021-01-15T19:08:06.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_thread_suicide_timeout": "150", 2021-01-15T19:08:06.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_op_thread_timeout": "15", 2021-01-15T19:08:06.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_open_classes_on_start": "true", 2021-01-15T19:08:06.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_os_flags": "0", 2021-01-15T19:08:06.264 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_peering_op_priority": "255", 2021-01-15T19:08:06.264 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_delete_cost": "1048576", 2021-01-15T19:08:06.264 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_delete_priority": "5", 2021-01-15T19:08:06.264 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2021-01-15T19:08:06.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2021-01-15T19:08:06.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_dups_tracked": "3000", 2021-01-15T19:08:06.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_trim_max": "10000", 2021-01-15T19:08:06.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_log_trim_min": "100", 2021-01-15T19:08:06.265 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2021-01-15T19:08:06.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_object_context_cache_count": "64", 2021-01-15T19:08:06.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pg_stat_report_interval_max": "500", 2021-01-15T19:08:06.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2021-01-15T19:08:06.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_min_evict_age": "0", 2021-01-15T19:08:06.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_min_flush_age": "0", 2021-01-15T19:08:06.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2021-01-15T19:08:06.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2021-01-15T19:08:06.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2021-01-15T19:08:06.267 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_crush_rule": "-1", 2021-01-15T19:08:06.268 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_ec_fast_read": "false", 2021-01-15T19:08:06.268 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2021-01-15T19:08:06.268 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_hashpspool": "true", 2021-01-15T19:08:06.269 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nodelete": "false", 2021-01-15T19:08:06.269 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nopgchange": "false", 2021-01-15T19:08:06.269 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flag_nosizechange": "false", 2021-01-15T19:08:06.270 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_flags": "0", 2021-01-15T19:08:06.270 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2021-01-15T19:08:06.270 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_min_size": "0", 2021-01-15T19:08:06.270 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2021-01-15T19:08:06.270 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pg_num": "32", 2021-01-15T19:08:06.271 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_pgp_num": "0", 2021-01-15T19:08:06.271 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2021-01-15T19:08:06.271 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_size": "2", 2021-01-15T19:08:06.271 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_default_type": "replicated", 2021-01-15T19:08:06.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2021-01-15T19:08:06.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pool_use_gmt_hitset": "true", 2021-01-15T19:08:06.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_push_per_object_cost": "1000", 2021-01-15T19:08:06.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_read_ec_check_for_errors": "false", 2021-01-15T19:08:06.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recover_clone_overlap": "true", 2021-01-15T19:08:06.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recover_clone_overlap_limit": "10", 2021-01-15T19:08:06.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_cost": "20971520", 2021-01-15T19:08:06.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_delay_start": "0.000000", 2021-01-15T19:08:06.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active": "0", 2021-01-15T19:08:06.274 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active_hdd": "3", 2021-01-15T19:08:06.274 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_active_ssd": "10", 2021-01-15T19:08:06.274 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_chunk": "8388608", 2021-01-15T19:08:06.274 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2021-01-15T19:08:06.275 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_max_single_start": "1", 2021-01-15T19:08:06.275 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_op_priority": "3", 2021-01-15T19:08:06.275 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_op_warn_multiple": "16", 2021-01-15T19:08:06.275 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_priority": "5", 2021-01-15T19:08:06.275 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_retry_interval": "30.000000", 2021-01-15T19:08:06.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep": "0.000000", 2021-01-15T19:08:06.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_hdd": "0.100000", 2021-01-15T19:08:06.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2021-01-15T19:08:06.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_recovery_sleep_ssd": "0.000000", 2021-01-15T19:08:06.276 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_repair_during_recovery": "false", 2021-01-15T19:08:06.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_requested_scrub_priority": "120", 2021-01-15T19:08:06.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_rollback_to_cluster_snap": "", 2021-01-15T19:08:06.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_auto_repair": "false", 2021-01-15T19:08:06.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_auto_repair_num_errors": "5", 2021-01-15T19:08:06.277 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_backoff_ratio": "0.660000", 2021-01-15T19:08:06.278 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_begin_hour": "0", 2021-01-15T19:08:06.278 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_begin_week_day": "0", 2021-01-15T19:08:06.278 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_chunk_max": "25", 2021-01-15T19:08:06.278 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_chunk_min": "5", 2021-01-15T19:08:06.278 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_cost": "52428800", 2021-01-15T19:08:06.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_during_recovery": "false", 2021-01-15T19:08:06.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_end_hour": "0", 2021-01-15T19:08:06.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_end_week_day": "0", 2021-01-15T19:08:06.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_extended_sleep": "0.000000", 2021-01-15T19:08:06.279 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2021-01-15T19:08:06.280 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_invalid_stats": "true", 2021-01-15T19:08:06.280 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_load_threshold": "0.500000", 2021-01-15T19:08:06.280 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_max_interval": "604800.000000", 2021-01-15T19:08:06.280 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_max_preemptions": "5", 2021-01-15T19:08:06.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_min_interval": "86400.000000", 2021-01-15T19:08:06.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_priority": "5", 2021-01-15T19:08:06.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_scrub_sleep": "0.000000", 2021-01-15T19:08:06.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_shutdown_pgref_assert": "false", 2021-01-15T19:08:06.281 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_skip_data_digest": "false", 2021-01-15T19:08:06.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_smart_report_timeout": "5", 2021-01-15T19:08:06.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_cost": "1048576", 2021-01-15T19:08:06.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_priority": "5", 2021-01-15T19:08:06.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep": "0.000000", 2021-01-15T19:08:06.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2021-01-15T19:08:06.283 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2021-01-15T19:08:06.283 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2021-01-15T19:08:06.283 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2021-01-15T19:08:06.283 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_target_transaction_size": "30", 2021-01-15T19:08:06.285 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_count": "4", 2021-01-15T19:08:06.285 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2021-01-15T19:08:06.285 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2021-01-15T19:08:06.286 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2021-01-15T19:08:06.286 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2021-01-15T19:08:06.287 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2021-01-15T19:08:06.287 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2021-01-15T19:08:06.288 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_default_cache_mode": "writeback", 2021-01-15T19:08:06.288 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2021-01-15T19:08:06.288 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tier_promote_max_objects_sec": "25", 2021-01-15T19:08:06.289 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_tracing": "false", 2021-01-15T19:08:06.289 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_use_stale_snap": "false", 2021-01-15T19:08:06.290 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2021-01-15T19:08:06.290 INFO:teuthology.orchestra.run.smithi001.stdout: "osdc_blkin_trace_all": "false", 2021-01-15T19:08:06.291 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_kill_at": "0", 2021-01-15T19:08:06.291 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_max_join_drift": "10", 2021-01-15T19:08:06.291 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_min": "500", 2021-01-15T19:08:06.292 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_min_wait": "0.050000", 2021-01-15T19:08:06.292 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_propose_interval": "1.000000", 2021-01-15T19:08:06.293 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_max": "500", 2021-01-15T19:08:06.293 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_service_trim_min": "10", 2021-01-15T19:08:06.293 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_stash_full_interval": "25", 2021-01-15T19:08:06.294 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_trim_max": "500", 2021-01-15T19:08:06.294 INFO:teuthology.orchestra.run.smithi001.stdout: "paxos_trim_min": "250", 2021-01-15T19:08:06.294 INFO:teuthology.orchestra.run.smithi001.stdout: "perf": "true", 2021-01-15T19:08:06.295 INFO:teuthology.orchestra.run.smithi001.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2021-01-15T19:08:06.295 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin_crypto_accelerator": "crypto_isal", 2021-01-15T19:08:06.295 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin_dir": "/usr/lib64/ceph", 2021-01-15T19:08:06.295 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "-", 2021-01-15T19:08:06.296 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrv": "", 2021-01-15T19:08:06.296 INFO:teuthology.orchestra.run.smithi001.stdout: "public_bind_addr": "-", 2021-01-15T19:08:06.297 INFO:teuthology.orchestra.run.smithi001.stdout: "public_network": "", 2021-01-15T19:08:06.297 INFO:teuthology.orchestra.run.smithi001.stdout: "public_network_interface": "", 2021-01-15T19:08:06.297 INFO:teuthology.orchestra.run.smithi001.stdout: "qat_compressor_enabled": "false", 2021-01-15T19:08:06.298 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_mon_op_timeout": "0", 2021-01-15T19:08:06.298 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_osd_op_timeout": "0", 2021-01-15T19:08:06.299 INFO:teuthology.orchestra.run.smithi001.stdout: "rados_tracing": "false", 2021-01-15T19:08:06.299 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_atime_update_interval": "60", 2021-01-15T19:08:06.299 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2021-01-15T19:08:06.300 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_balance_parent_reads": "false", 2021-01-15T19:08:06.300 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_balance_snap_reads": "false", 2021-01-15T19:08:06.300 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blkin_trace_all": "false", 2021-01-15T19:08:06.301 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blocklist_expire_seconds": "0", 2021-01-15T19:08:06.301 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_blocklist_on_break_lock": "true", 2021-01-15T19:08:06.301 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache": "true", 2021-01-15T19:08:06.302 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_block_writes_upfront": "false", 2021-01-15T19:08:06.302 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty": "25165824", 2021-01-15T19:08:06.303 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty_age": "1.000000", 2021-01-15T19:08:06.303 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_max_dirty_object": "0", 2021-01-15T19:08:06.303 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_policy": "writearound", 2021-01-15T19:08:06.303 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_size": "33554432", 2021-01-15T19:08:06.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_target_dirty": "16777216", 2021-01-15T19:08:06.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_cache_writethrough_until_flush": "true", 2021-01-15T19:08:06.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_clone_copy_on_read": "false", 2021-01-15T19:08:06.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_compression_hint": "none", 2021-01-15T19:08:06.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_concurrent_management_ops": "10", 2021-01-15T19:08:06.305 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_config_pool_override_update_timestamp": "0", 2021-01-15T19:08:06.305 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_clone_format": "auto", 2021-01-15T19:08:06.305 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_data_pool": "", 2021-01-15T19:08:06.305 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_features": "61", 2021-01-15T19:08:06.305 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_format": "2", 2021-01-15T19:08:06.306 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_map_options": "", 2021-01-15T19:08:06.306 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_order": "22", 2021-01-15T19:08:06.306 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_pool": "rbd", 2021-01-15T19:08:06.306 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2021-01-15T19:08:06.307 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_stripe_count": "0", 2021-01-15T19:08:06.307 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_default_stripe_unit": "0", 2021-01-15T19:08:06.307 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_disable_zero_copy_writes": "true", 2021-01-15T19:08:06.307 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_discard_granularity_bytes": "65536", 2021-01-15T19:08:06.307 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_discard_on_zeroed_write_same": "true", 2021-01-15T19:08:06.308 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_enable_alloc_hint": "true", 2021-01-15T19:08:06.308 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2021-01-15T19:08:06.308 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_io_scheduler": "simple", 2021-01-15T19:08:06.308 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2021-01-15T19:08:06.309 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_commit_age": "5.000000", 2021-01-15T19:08:06.309 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2021-01-15T19:08:06.309 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_max_payload_bytes": "16384", 2021-01-15T19:08:06.309 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_age": "0.000000", 2021-01-15T19:08:06.310 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_bytes": "1048576", 2021-01-15T19:08:06.310 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_flush_interval": "0", 2021-01-15T19:08:06.311 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2021-01-15T19:08:06.311 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2021-01-15T19:08:06.311 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_order": "24", 2021-01-15T19:08:06.312 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_pool": "", 2021-01-15T19:08:06.312 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_journal_splay_width": "4", 2021-01-15T19:08:06.313 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_localize_parent_reads": "false", 2021-01-15T19:08:06.313 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_localize_snap_reads": "false", 2021-01-15T19:08:06.313 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2021-01-15T19:08:06.314 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2021-01-15T19:08:06.314 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2021-01-15T19:08:06.314 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2021-01-15T19:08:06.315 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2021-01-15T19:08:06.315 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2021-01-15T19:08:06.315 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_type": "simple", 2021-01-15T19:08:06.316 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2021-01-15T19:08:06.316 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_image_state_check_interval": "30", 2021-01-15T19:08:06.316 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2021-01-15T19:08:06.317 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2021-01-15T19:08:06.317 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2021-01-15T19:08:06.317 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2021-01-15T19:08:06.317 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2021-01-15T19:08:06.318 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_autotune": "true", 2021-01-15T19:08:06.318 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_base": "805306368", 2021-01-15T19:08:06.318 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2021-01-15T19:08:06.318 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_min": "134217728", 2021-01-15T19:08:06.318 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2021-01-15T19:08:06.319 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2021-01-15T19:08:06.319 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_memory_target": "4294967296", 2021-01-15T19:08:06.319 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_perf_stats_prio": "5", 2021-01-15T19:08:06.319 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2021-01-15T19:08:06.319 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2021-01-15T19:08:06.320 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_delete_delay": "0", 2021-01-15T19:08:06.320 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_max_mirroring_snapshots": "3", 2021-01-15T19:08:06.320 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_replay_delay": "0", 2021-01-15T19:08:06.320 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2021-01-15T19:08:06.320 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2021-01-15T19:08:06.321 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_to_trash_on_remove": "false", 2021-01-15T19:08:06.321 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2021-01-15T19:08:06.321 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_mtime_update_interval": "60", 2021-01-15T19:08:06.321 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_non_blocking_aio": "true", 2021-01-15T19:08:06.321 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_op_thread_timeout": "60", 2021-01-15T19:08:06.322 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_op_threads": "1", 2021-01-15T19:08:06.322 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_parent_cache_enabled": "false", 2021-01-15T19:08:06.322 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_log_periodic_stats": "false", 2021-01-15T19:08:06.322 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_mode": "disabled", 2021-01-15T19:08:06.323 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_path": "/tmp", 2021-01-15T19:08:06.323 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_persistent_cache_size": "1073741824", 2021-01-15T19:08:06.323 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_plugins": "", 2021-01-15T19:08:06.323 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_burst": "0", 2021-01-15T19:08:06.324 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_burst_seconds": "1", 2021-01-15T19:08:06.324 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_bps_limit": "0", 2021-01-15T19:08:06.324 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_burst": "0", 2021-01-15T19:08:06.325 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_burst_seconds": "1", 2021-01-15T19:08:06.325 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_iops_limit": "0", 2021-01-15T19:08:06.325 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_burst": "0", 2021-01-15T19:08:06.325 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2021-01-15T19:08:06.325 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_bps_limit": "0", 2021-01-15T19:08:06.326 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_burst": "0", 2021-01-15T19:08:06.326 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2021-01-15T19:08:06.326 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_read_iops_limit": "0", 2021-01-15T19:08:06.326 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_schedule_tick_min": "50", 2021-01-15T19:08:06.327 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_burst": "0", 2021-01-15T19:08:06.327 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2021-01-15T19:08:06.327 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_bps_limit": "0", 2021-01-15T19:08:06.328 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_burst": "0", 2021-01-15T19:08:06.328 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2021-01-15T19:08:06.329 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_qos_write_iops_limit": "0", 2021-01-15T19:08:06.329 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_quiesce_notification_attempts": "10", 2021-01-15T19:08:06.329 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_read_from_replica_policy": "default", 2021-01-15T19:08:06.329 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2021-01-15T19:08:06.330 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_max_bytes": "524288", 2021-01-15T19:08:06.330 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_readahead_trigger_requests": "10", 2021-01-15T19:08:06.330 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_request_timed_out_seconds": "30", 2021-01-15T19:08:06.331 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_skip_partial_discard": "true", 2021-01-15T19:08:06.331 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2021-01-15T19:08:06.331 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_tracing": "false", 2021-01-15T19:08:06.332 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_validate_names": "true", 2021-01-15T19:08:06.332 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_validate_pool": "true", 2021-01-15T19:08:06.332 INFO:teuthology.orchestra.run.smithi001.stdout: "restapi_base_url": "", 2021-01-15T19:08:06.332 INFO:teuthology.orchestra.run.smithi001.stdout: "restapi_log_level": "", 2021-01-15T19:08:06.332 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_acl_grants_max_num": "100", 2021-01-15T19:08:06.333 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_admin_entry": "admin", 2021-01-15T19:08:06.333 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_barbican_url": "", 2021-01-15T19:08:06.333 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2021-01-15T19:08:06.333 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_default_quota_max_size": "-1", 2021-01-15T19:08:06.334 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_index_max_aio": "128", 2021-01-15T19:08:06.334 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_cache_size": "10000", 2021-01-15T19:08:06.334 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_soft_threshold": "0.950000", 2021-01-15T19:08:06.334 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_bucket_quota_ttl": "600", 2021-01-15T19:08:06.335 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_enabled": "true", 2021-01-15T19:08:06.335 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_expiry_interval": "900", 2021-01-15T19:08:06.335 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cache_lru_size": "10000", 2021-01-15T19:08:06.335 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_content_length_compat": "false", 2021-01-15T19:08:06.335 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_copy_obj_progress": "true", 2021-01-15T19:08:06.336 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2021-01-15T19:08:06.336 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cors_rules_max_num": "100", 2021-01-15T19:08:06.336 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_cross_domain_policy": "", 2021-01-15T19:08:06.336 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_default_encryption_key": "", 2021-01-15T19:08:06.336 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_require_ssl": "true", 2021-01-15T19:08:06.337 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2021-01-15T19:08:06.337 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2021-01-15T19:08:06.337 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_suppress_logs": "true", 2021-01-15T19:08:06.338 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_addr": "", 2021-01-15T19:08:06.338 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_auth": "token", 2021-01-15T19:08:06.338 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_namespace": "", 2021-01-15T19:08:06.338 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_prefix": "", 2021-01-15T19:08:06.339 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_secret_engine": "transit", 2021-01-15T19:08:06.339 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_crypt_vault_token_file": "", 2021-01-15T19:08:06.339 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_buffersize": "524288", 2021-01-15T19:08:06.339 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_low_speed_limit": "1024", 2021-01-15T19:08:06.339 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_low_speed_time": "300", 2021-01-15T19:08:06.340 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_curl_wait_timeout_ms": "1000", 2021-01-15T19:08:06.340 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2021-01-15T19:08:06.340 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_backing": "auto", 2021-01-15T19:08:06.340 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_changes_size": "1000", 2021-01-15T19:08:06.340 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_num_shards": "128", 2021-01-15T19:08:06.341 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_obj_prefix": "data_log", 2021-01-15T19:08:06.341 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_log_window": "30", 2021-01-15T19:08:06.341 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_data_notify_interval_msec": "200", 2021-01-15T19:08:06.342 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_realm_info_oid": "default.realm", 2021-01-15T19:08:06.342 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_region_info_oid": "default.region", 2021-01-15T19:08:06.342 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_zone_info_oid": "default.zone", 2021-01-15T19:08:06.342 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2021-01-15T19:08:06.343 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_defer_to_bucket_acls": "", 2021-01-15T19:08:06.343 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_delete_multi_obj_max_num": "1000", 2021-01-15T19:08:06.343 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_lim": "0.000000", 2021-01-15T19:08:06.343 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_res": "100.000000", 2021-01-15T19:08:06.343 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2021-01-15T19:08:06.344 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_lim": "0.000000", 2021-01-15T19:08:06.344 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_res": "200.000000", 2021-01-15T19:08:06.344 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2021-01-15T19:08:06.344 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_lim": "0.000000", 2021-01-15T19:08:06.345 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_res": "500.000000", 2021-01-15T19:08:06.345 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_data_wgt": "500.000000", 2021-01-15T19:08:06.345 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2021-01-15T19:08:06.345 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_res": "500.000000", 2021-01-15T19:08:06.345 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2021-01-15T19:08:06.346 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dns_name": "", 2021-01-15T19:08:06.346 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dns_s3website_name": "", 2021-01-15T19:08:06.346 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_dynamic_resharding": "true", 2021-01-15T19:08:06.346 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, pubsub", 2021-01-15T19:08:06.346 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_gc_threads": "true", 2021-01-15T19:08:06.347 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_lc_threads": "true", 2021-01-15T19:08:06.347 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_ops_log": "false", 2021-01-15T19:08:06.347 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_quota_threads": "true", 2021-01-15T19:08:06.347 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_static_website": "false", 2021-01-15T19:08:06.347 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enable_usage_log": "false", 2021-01-15T19:08:06.348 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_enforce_swift_acls": "true", 2021-01-15T19:08:06.348 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_exit_timeout_secs": "120", 2021-01-15T19:08:06.348 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_expose_bucket": "false", 2021-01-15T19:08:06.349 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_extended_http_attrs": "", 2021-01-15T19:08:06.349 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_fcgi_socket_backlog": "1024", 2021-01-15T19:08:06.349 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2021-01-15T19:08:06.349 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_frontends": "beast port=7480", 2021-01-15T19:08:06.349 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_concurrent_io": "10", 2021-01-15T19:08:06.350 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_deferred": "50", 2021-01-15T19:08:06.350 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2021-01-15T19:08:06.350 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_objs": "32", 2021-01-15T19:08:06.350 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_queue_size": "134213632", 2021-01-15T19:08:06.350 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_max_trim_chunk": "16", 2021-01-15T19:08:06.351 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_obj_min_wait": "7200", 2021-01-15T19:08:06.351 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_processor_max_time": "3600", 2021-01-15T19:08:06.351 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_gc_processor_period": "3600", 2021-01-15T19:08:06.351 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_get_obj_max_req_size": "4194304", 2021-01-15T19:08:06.352 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_get_obj_window_size": "16777216", 2021-01-15T19:08:06.352 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_healthcheck_disabling_path": "", 2021-01-15T19:08:06.352 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_host": "", 2021-01-15T19:08:06.353 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ignore_get_invalid_range": "false", 2021-01-15T19:08:06.353 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_init_timeout": "300", 2021-01-15T19:08:06.353 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2021-01-15T19:08:06.353 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_admin_roles": "", 2021-01-15T19:08:06.354 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2021-01-15T19:08:06.354 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_domain": "", 2021-01-15T19:08:06.354 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_password": "", 2021-01-15T19:08:06.354 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_password_path": "", 2021-01-15T19:08:06.355 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_project": "", 2021-01-15T19:08:06.355 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_tenant": "", 2021-01-15T19:08:06.355 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_token": "", 2021-01-15T19:08:06.356 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_token_path": "", 2021-01-15T19:08:06.356 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_admin_user": "", 2021-01-15T19:08:06.356 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_api_version": "2", 2021-01-15T19:08:06.356 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_domain": "", 2021-01-15T19:08:06.357 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_password": "", 2021-01-15T19:08:06.357 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_project": "", 2021-01-15T19:08:06.357 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_tenant": "", 2021-01-15T19:08:06.357 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_barbican_user": "", 2021-01-15T19:08:06.358 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_implicit_tenants": "false", 2021-01-15T19:08:06.358 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_token_cache_size": "10000", 2021-01-15T19:08:06.358 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_url": "", 2021-01-15T19:08:06.358 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_keystone_verify_ssl": "true", 2021-01-15T19:08:06.358 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_debug_interval": "-1", 2021-01-15T19:08:06.359 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_lock_max_time": "90", 2021-01-15T19:08:06.359 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_objs": "32", 2021-01-15T19:08:06.359 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_rules": "1000", 2021-01-15T19:08:06.359 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_worker": "3", 2021-01-15T19:08:06.359 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_max_wp_worker": "3", 2021-01-15T19:08:06.360 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lc_thread_delay": "0", 2021-01-15T19:08:06.360 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2021-01-15T19:08:06.360 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_dnattr": "uid", 2021-01-15T19:08:06.360 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2021-01-15T19:08:06.360 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_searchfilter": "", 2021-01-15T19:08:06.361 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2021-01-15T19:08:06.361 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ldap_uri": "ldaps://", 2021-01-15T19:08:06.361 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2021-01-15T19:08:06.361 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_list_bucket_min_readahead": "1000", 2021-01-15T19:08:06.361 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_list_buckets_max_chunk": "1000", 2021-01-15T19:08:06.362 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_http_headers": "", 2021-01-15T19:08:06.362 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_nonexistent_bucket": "false", 2021-01-15T19:08:06.362 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2021-01-15T19:08:06.362 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_log_object_name_utc": "false", 2021-01-15T19:08:06.362 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2021-01-15T19:08:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attr_name_len": "0", 2021-01-15T19:08:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attr_size": "0", 2021-01-15T19:08:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_attrs_num_in_req": "0", 2021-01-15T19:08:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_chunk_size": "4194304", 2021-01-15T19:08:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_concurrent_requests": "1024", 2021-01-15T19:08:06.364 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_dynamic_shards": "1999", 2021-01-15T19:08:06.364 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_listing_results": "1000", 2021-01-15T19:08:06.364 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_notify_retries": "3", 2021-01-15T19:08:06.364 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_objs_per_shard": "100000", 2021-01-15T19:08:06.364 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_put_param_size": "1048576", 2021-01-15T19:08:06.365 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_put_size": "5368709120", 2021-01-15T19:08:06.365 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_max_slo_entries": "1000", 2021-01-15T19:08:06.365 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_md_log_max_shards": "64", 2021-01-15T19:08:06.365 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_md_notify_interval_msec": "200", 2021-01-15T19:08:06.365 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_mime_types_file": "/etc/mime.types", 2021-01-15T19:08:06.366 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_mp_lock_max_time": "600", 2021-01-15T19:08:06.366 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multipart_min_part_size": "5242880", 2021-01-15T19:08:06.366 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_multipart_part_upload_limit": "10000", 2021-01-15T19:08:06.366 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_fhcache_partitions": "3", 2021-01-15T19:08:06.366 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_fhcache_size": "2017", 2021-01-15T19:08:06.367 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2021-01-15T19:08:06.367 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_lru_lanes": "5", 2021-01-15T19:08:06.367 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_max_gc": "300", 2021-01-15T19:08:06.367 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_namespace_expire_secs": "300", 2021-01-15T19:08:06.367 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_gc_threads": "false", 2021-01-15T19:08:06.368 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_lc_threads": "false", 2021-01-15T19:08:06.368 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_quota_threads": "false", 2021-01-15T19:08:06.368 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_run_sync_thread": "false", 2021-01-15T19:08:06.368 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_s3_fast_attrs": "false", 2021-01-15T19:08:06.368 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_nfs_write_completion_interval_s": "10", 2021-01-15T19:08:06.369 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_async_rados_threads": "32", 2021-01-15T19:08:06.369 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_control_oids": "8", 2021-01-15T19:08:06.369 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_num_rados_handles": "1", 2021-01-15T19:08:06.369 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_numa_node": "-1", 2021-01-15T19:08:06.369 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_obj_stripe_size": "4194304", 2021-01-15T19:08:06.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_obj_tombstone_cache_size": "1000", 2021-01-15T19:08:06.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_chunk_size": "100", 2021-01-15T19:08:06.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_gc_interval": "600", 2021-01-15T19:08:06.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_objexp_hints_num_shards": "127", 2021-01-15T19:08:06.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_olh_pending_timeout_sec": "3600", 2021-01-15T19:08:06.371 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_thread_suicide_timeout": "0", 2021-01-15T19:08:06.371 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_thread_timeout": "600", 2021-01-15T19:08:06.371 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_op_tracing": "false", 2021-01-15T19:08:06.371 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_token": "", 2021-01-15T19:08:06.371 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_url": "", 2021-01-15T19:08:06.372 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_opa_verify_ssl": "true", 2021-01-15T19:08:06.372 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_data_backlog": "5242880", 2021-01-15T19:08:06.372 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_rados": "true", 2021-01-15T19:08:06.372 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_ops_log_socket_path": "", 2021-01-15T19:08:06.372 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_override_bucket_index_max_shards": "0", 2021-01-15T19:08:06.373 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2021-01-15T19:08:06.373 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_push_interval": "2.000000", 2021-01-15T19:08:06.373 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_push_interval_max": "30.000000", 2021-01-15T19:08:06.373 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_period_root_pool": ".rgw.root", 2021-01-15T19:08:06.373 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_port": "", 2021-01-15T19:08:06.374 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_print_continue": "true", 2021-01-15T19:08:06.374 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_print_prohibited_content_length": "false", 2021-01-15T19:08:06.374 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_put_obj_max_window_size": "67108864", 2021-01-15T19:08:06.374 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_put_obj_min_window_size": "16777216", 2021-01-15T19:08:06.374 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2021-01-15T19:08:06.375 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_pg_num_min": "8", 2021-01-15T19:08:06.375 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_pool_recovery_priority": "5", 2021-01-15T19:08:06.375 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rados_tracing": "false", 2021-01-15T19:08:06.375 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm": "", 2021-01-15T19:08:06.375 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_realm_root_pool": ".rgw.root", 2021-01-15T19:08:06.376 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_region": "", 2021-01-15T19:08:06.376 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_region_root_pool": ".rgw.root", 2021-01-15T19:08:06.376 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_relaxed_region_enforcement": "false", 2021-01-15T19:08:06.376 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_relaxed_s3_bucket_names": "false", 2021-01-15T19:08:06.376 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2021-01-15T19:08:06.377 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_request_uri": "", 2021-01-15T19:08:06.377 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_batch_size": "64", 2021-01-15T19:08:06.377 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_bucket_lock_duration": "360", 2021-01-15T19:08:06.377 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_max_aio": "128", 2021-01-15T19:08:06.377 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_num_logs": "16", 2021-01-15T19:08:06.378 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_reshard_thread_interval": "600", 2021-01-15T19:08:06.378 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_resolve_cname": "false", 2021-01-15T19:08:06.378 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_rest_getusage_op_compat": "false", 2021-01-15T19:08:06.378 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_run_sync_thread": "true", 2021-01-15T19:08:06.378 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_order": "sts, external, local", 2021-01-15T19:08:06.379 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_keystone": "false", 2021-01-15T19:08:06.379 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_ldap": "false", 2021-01-15T19:08:06.379 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_rados": "true", 2021-01-15T19:08:06.379 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_auth_use_sts": "false", 2021-01-15T19:08:06.379 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_s3_success_create_obj_status": "0", 2021-01-15T19:08:06.380 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_safe_max_objects_per_shard": "102400", 2021-01-15T19:08:06.380 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_scheduler_type": "throttler", 2021-01-15T19:08:06.380 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_script_uri": "", 2021-01-15T19:08:06.380 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_service_provider_name": "", 2021-01-15T19:08:06.380 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_shard_warning_threshold": "90.000000", 2021-01-15T19:08:06.381 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_socket_path": "", 2021-01-15T19:08:06.381 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_client_id": "", 2021-01-15T19:08:06.381 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_client_secret": "", 2021-01-15T19:08:06.381 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_entry": "sts", 2021-01-15T19:08:06.381 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_key": "sts", 2021-01-15T19:08:06.382 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_max_session_duration": "43200", 2021-01-15T19:08:06.382 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_min_session_duration": "900", 2021-01-15T19:08:06.382 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sts_token_introspection_url": "", 2021-01-15T19:08:06.382 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_account_in_url": "false", 2021-01-15T19:08:06.382 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_auth_entry": "auth", 2021-01-15T19:08:06.383 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_auth_url": "", 2021-01-15T19:08:06.383 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_custom_header": "", 2021-01-15T19:08:06.383 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_enforce_content_length": "false", 2021-01-15T19:08:06.383 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_need_stats": "true", 2021-01-15T19:08:06.383 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_tenant_name": "", 2021-01-15T19:08:06.384 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_token_expiration": "86400", 2021-01-15T19:08:06.384 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_url": "", 2021-01-15T19:08:06.384 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_url_prefix": "swift", 2021-01-15T19:08:06.384 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_swift_versioning_enabled": "false", 2021-01-15T19:08:06.384 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2021-01-15T19:08:06.385 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_lease_period": "120", 2021-01-15T19:08:06.385 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2021-01-15T19:08:06.385 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_interval": "1200", 2021-01-15T19:08:06.385 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_max_buckets": "16", 2021-01-15T19:08:06.385 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2021-01-15T19:08:06.386 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2021-01-15T19:08:06.386 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_obj_etag_verify": "false", 2021-01-15T19:08:06.386 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_history_size": "4096", 2021-01-15T19:08:06.386 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_per_node_log_size": "32", 2021-01-15T19:08:06.386 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2021-01-15T19:08:06.387 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_thread_pool_size": "512", 2021-01-15T19:08:06.387 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_comment": "", 2021-01-15T19:08:06.387 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_createby": "", 2021-01-15T19:08:06.387 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_encoding": "", 2021-01-15T19:08:06.387 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_flag": "false", 2021-01-15T19:08:06.388 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_origin": "", 2021-01-15T19:08:06.388 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_sha_unit": "524288", 2021-01-15T19:08:06.388 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_torrent_tracker": "", 2021-01-15T19:08:06.388 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_trust_forwarded_https": "false", 2021-01-15T19:08:06.388 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_log_flush_threshold": "1024", 2021-01-15T19:08:06.389 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_log_tick_interval": "30", 2021-01-15T19:08:06.389 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_max_shards": "32", 2021-01-15T19:08:06.389 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_usage_max_user_shards": "1", 2021-01-15T19:08:06.389 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_use_opa_authz": "false", 2021-01-15T19:08:06.389 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_default_quota_max_objects": "-1", 2021-01-15T19:08:06.390 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_default_quota_max_size": "-1", 2021-01-15T19:08:06.390 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_max_buckets": "1000", 2021-01-15T19:08:06.390 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2021-01-15T19:08:06.390 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_idle_users": "false", 2021-01-15T19:08:06.390 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_interval": "86400", 2021-01-15T19:08:06.391 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_quota_sync_wait_time": "86400", 2021-01-15T19:08:06.391 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_user_unique_email": "true", 2021-01-15T19:08:06.391 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_verify_ssl": "true", 2021-01-15T19:08:06.391 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_website_routing_rules_max_num": "50", 2021-01-15T19:08:06.391 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone": "", 2021-01-15T19:08:06.392 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zone_root_pool": ".rgw.root", 2021-01-15T19:08:06.392 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup": "", 2021-01-15T19:08:06.392 INFO:teuthology.orchestra.run.smithi001.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2021-01-15T19:08:06.392 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_block_size": "4096", 2021-01-15T19:08:06.392 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_bloom_bits_per_key": "20", 2021-01-15T19:08:06.393 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2021-01-15T19:08:06.393 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "true", 2021-01-15T19:08:06.393 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_row_ratio": "0.000000", 2021-01-15T19:08:06.393 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_shard_bits": "4", 2021-01-15T19:08:06.393 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_size": "536870912", 2021-01-15T19:08:06.394 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_cache_type": "binned_lru", 2021-01-15T19:08:06.394 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_compaction_stats": "false", 2021-01-15T19:08:06.394 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_extended_stats": "false", 2021-01-15T19:08:06.394 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_collect_memory_stats": "false", 2021-01-15T19:08:06.394 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_delete_range_threshold": "1048576", 2021-01-15T19:08:06.395 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_index_type": "binary_search", 2021-01-15T19:08:06.395 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_log_to_ceph_log": "true", 2021-01-15T19:08:06.395 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_metadata_block_size": "4096", 2021-01-15T19:08:06.395 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_partition_filters": "false", 2021-01-15T19:08:06.395 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_perf": "false", 2021-01-15T19:08:06.396 INFO:teuthology.orchestra.run.smithi001.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2021-01-15T19:08:06.396 INFO:teuthology.orchestra.run.smithi001.stdout: "rotating_keys_bootstrap_timeout": "30", 2021-01-15T19:08:06.396 INFO:teuthology.orchestra.run.smithi001.stdout: "rotating_keys_renewal_timeout": "10", 2021-01-15T19:08:06.396 INFO:teuthology.orchestra.run.smithi001.stdout: "run_dir": "/var/run/ceph", 2021-01-15T19:08:06.396 INFO:teuthology.orchestra.run.smithi001.stdout: "setgroup": "", 2021-01-15T19:08:06.397 INFO:teuthology.orchestra.run.smithi001.stdout: "setuser": "", 2021-01-15T19:08:06.397 INFO:teuthology.orchestra.run.smithi001.stdout: "setuser_match_path": "", 2021-01-15T19:08:06.397 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_misplaced_ratio": "0.050000", 2021-01-15T19:08:06.397 INFO:teuthology.orchestra.run.smithi001.stdout: "thp": "false", 2021-01-15T19:08:06.397 INFO:teuthology.orchestra.run.smithi001.stdout: "threadpool_default_timeout": "60", 2021-01-15T19:08:06.398 INFO:teuthology.orchestra.run.smithi001.stdout: "threadpool_empty_queue_max_wait": "2", 2021-01-15T19:08:06.398 INFO:teuthology.orchestra.run.smithi001.stdout: "throttler_perf_counter": "true" 2021-01-15T19:08:06.398 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:06.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2021-01-15T19:08:06.543 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a: {} 2021-01-15T19:08:06.555 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b: {} 2021-01-15T19:08:06.565 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c: {} 2021-01-15T19:08:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2021-01-15T19:08:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2021-01-15T19:08:06.566 INFO:teuthology.orchestra.run.smithi001.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2021-01-15T19:08:06.577 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2021-01-15T19:08:06.594 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2021-01-15T19:08:06.595 DEBUG:teuthology.orchestra.run.smithi001:> type ceph-objectstore-tool 2021-01-15T19:08:06.611 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2021-01-15T19:08:06.612 INFO:teuthology.run_tasks:Running task rados... 2021-01-15T19:08:06.641 INFO:tasks.rados:Beginning rados... 2021-01-15T19:08:06.642 DEBUG:teuthology.run_tasks:Unwinding manager rados 2021-01-15T19:08:06.656 INFO:tasks.rados:joining rados 2021-01-15T19:08:06.656 INFO:tasks.thrashosds.thrasher:starting do_thrash 2021-01-15T19:08:06.657 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:06.657 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:06.657 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 1.000000 2021-01-15T19:08:06.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2021-01-15T19:08:06.659 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2021-01-15T19:08:06.660 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2021-01-15T19:08:06.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:06.661 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2021-01-15T19:08:06.661 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:06.662 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2021-01-15T19:08:06.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:08:06.664 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2021-01-15T19:08:06.664 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set isaprofile crush-failure-domain=osd k=2 m=1 name=isaprofile plugin=isa technique=reed_sol_van 2021-01-15T19:08:06.665 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:06.659+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:06.762 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:06.760+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:06.863 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:06.861+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:06.881 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:06.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:06.881 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:06.881 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:06.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:06.899 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:06.924 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:06.950 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:06.964 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:06.962+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:06.972 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:07.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:07.051 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:07.065 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:07.064+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:07.076 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:07.076 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:07.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:07.091 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:07.130 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:07.131 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.133 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:07.166 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:07.165+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:07.209 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:08:07.213 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.7 primary-affinity to 1 (8655362) 2021-01-15T19:08:07.222 INFO:tasks.rados:starting run 0 out of 1 2021-01-15T19:08:07.223 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2021-01-15T19:08:07.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure isaprofile 2021-01-15T19:08:07.228 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:07.228 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:07.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:07.267 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:07.266+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:07.367 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:07.366+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:07.421 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:07.422 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:07.434 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:07.470 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:07.468+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:07.570 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:07.569+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:07.605 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:07.606 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:07.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:07.671 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:07.670+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:07.749 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:07.750 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:07.761 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:07.773 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:07.772+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:07.878 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:07.873+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:07.934 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:07.935 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:07.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:07.990 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:07.988+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:08.077 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:08.076+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:08.120 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:08.121 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:08.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:08.216 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:08.214+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:08.278 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:08.277+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:08.306 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:08.306 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:08.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:08.400 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:08.399+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:08.481 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:08.479+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:08.491 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:08.491 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:08.498 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:08.582 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:08.580+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:08.669 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:08.670 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:08.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:08.684 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:08.682+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:08.784 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:08.783+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:08.851 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:08.851 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:08.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:08.903 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:08.902+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:08.986 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:08.984+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:08.998 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:08.998 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:09.010 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:09.093 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:09.092+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:09.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:09.174 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:09.174 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:09.183 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:09.188 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:09.186+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:09.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:08:09.226 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'unique_pool_0' created 2021-01-15T19:08:09.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2021-01-15T19:08:09.289 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:09.287+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:09.309 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:09.343 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:09.351 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:09.351 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:09.360 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:09.364 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:09.387 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:09.401 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:09.400+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:09.408 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:09.434 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:09.454 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:09.475 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:09.476 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.476 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:09.529 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:09.530 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:09.530 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:09.527+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:09.538 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:09.591 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:09.589+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:09.662 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:09.662 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:09.663 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:09.663 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:09.663 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:09.672 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:09.715 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:09.714+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:09.792 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:09.791+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:09.796 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:09.797 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:09.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:09.797 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:09.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.798 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4245.0:2 1.0 e14/13 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head v 14'1, mlcod=0'0)", 2021-01-15T19:08:09.798 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:07:51.931673+0000", 2021-01-15T19:08:09.798 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.864197300000001, 2021-01-15T19:08:09.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082302700000000005, 2021-01-15T19:08:09.798 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:09.799 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:09.799 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:09.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:09.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931673+0000", 2021-01-15T19:08:09.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:09.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:09.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931673+0000", 2021-01-15T19:08:09.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4565000000000003e-05 2021-01-15T19:08:09.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:09.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931737+0000", 2021-01-15T19:08:09.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9440000000000005e-06 2021-01-15T19:08:09.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:09.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931745+0000", 2021-01-15T19:08:09.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0482999999999999e-05 2021-01-15T19:08:09.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:09.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931756+0000", 2021-01-15T19:08:09.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1270000000000003e-06 2021-01-15T19:08:09.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:09.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931764+0000", 2021-01-15T19:08:09.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7147000000000001e-05 2021-01-15T19:08:09.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:09.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931811+0000", 2021-01-15T19:08:09.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7387000000000001e-05 2021-01-15T19:08:09.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:09.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931839+0000", 2021-01-15T19:08:09.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060284300000000004 2021-01-15T19:08:09.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:09.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.932441+0000", 2021-01-15T19:08:09.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530999999999997e-05 2021-01-15T19:08:09.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:09.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.932496+0000", 2021-01-15T19:08:09.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082302700000000005 2021-01-15T19:08:09.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:09.810 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:09.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:09.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4245.0:4 1.0 e14/13 1:916bc198:::INTEL_SSDPEDMX020T4_CVPD514100122P0SGN:head v 14'2, mlcod=0'0)", 2021-01-15T19:08:09.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:07:52.012390+0000", 2021-01-15T19:08:09.811 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.783479771, 2021-01-15T19:08:09.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015440479999999999, 2021-01-15T19:08:09.811 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:09.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:09.812 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:09.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:09.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012390+0000", 2021-01-15T19:08:09.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:09.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:09.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012390+0000", 2021-01-15T19:08:09.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1465000000000003e-05 2021-01-15T19:08:09.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:09.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012442+0000", 2021-01-15T19:08:09.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1226999999999999e-05 2021-01-15T19:08:09.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:09.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012453+0000", 2021-01-15T19:08:09.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4151000000000001e-05 2021-01-15T19:08:09.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:09.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012467+0000", 2021-01-15T19:08:09.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4469999999999995e-06 2021-01-15T19:08:09.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:09.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012477+0000", 2021-01-15T19:08:09.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5005000000000004e-05 2021-01-15T19:08:09.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:09.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012562+0000", 2021-01-15T19:08:09.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2855000000000001e-05 2021-01-15T19:08:09.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:09.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012585+0000", 2021-01-15T19:08:09.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012942889999999999 2021-01-15T19:08:09.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:09.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.013879+0000", 2021-01-15T19:08:09.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5609000000000002e-05 2021-01-15T19:08:09.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:09.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:09.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:09.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.013934+0000", 2021-01-15T19:08:09.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015440479999999999 2021-01-15T19:08:09.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:09.822 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:09.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:09.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:09.822 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:09.823 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:09.824 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:09.907 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:09.906+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:09.990 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:09.990 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:09.990 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:09.991 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:09.994 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:09.993+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:09.999 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:10.095 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:10.094+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:10.164 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:10.165 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:10.165 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:10.165 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:10.165 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:10.174 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:10.196 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:10.195+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:10.231 INFO:teuthology.orchestra.run.smithi001.stderr:enabled application 'rados' on pool 'unique_pool_0' 2021-01-15T19:08:10.235 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:08:10.297 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:10.295+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:10.344 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:10.345 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:10.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:10.345 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:08:10.345 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:10.354 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:10.437 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:10.436+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:10.498 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:10.497+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:10.518 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:10.519 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:10.519 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:10.519 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:10.528 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:10.599 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:10.597+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:10.691 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:10.691 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:10.691 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:10.691 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:10.692 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:10.700 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:10.701 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:10.698+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:10.801 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:10.799+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:10.866 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:10.866 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:10.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:10.867 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:08:10.868 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:10.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:10.901 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:10.900+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:11.002 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:11.001+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:11.022 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:11.022 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:11.022 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:11.023 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:11.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:11.104 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:11.103+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:11.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 min_size 2 2021-01-15T19:08:11.299 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:11.300 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:11.300 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:11.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:11.301 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:11.299+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:11.399 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:11.398+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:11.451 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:11.452 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [] 2021-01-15T19:08:11.454 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:11.464 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:11.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:11.507 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:11.504+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:11.603 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:11.601+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:11.659 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:11.664 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:11.665 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:11.665 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:11.675 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:11.678 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:11.698 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:11.707 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:11.705+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:11.719 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:11.740 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:11.761 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:11.782 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:11.803 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:11.804 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.804 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:11.809 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:11.808+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:11.815 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:11.816 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:11.826 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:11.911 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:11.909+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:12.006 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.006 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:12.018 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:12.013+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:12.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:12.115 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:12.113+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:12.192 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.192 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:12.203 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:12.229 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:12.230 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:12.230 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 3 min_size to 2 2021-01-15T19:08:12.236 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:12.232+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:12.246 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2021-01-15T19:08:12.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:08:12.312 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight read -> 100 2021-01-15T19:08:12.313 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight delete -> 50 2021-01-15T19:08:12.314 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight snap_create -> 50 2021-01-15T19:08:12.314 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight snap_remove -> 50 2021-01-15T19:08:12.315 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight rollback -> 50 2021-01-15T19:08:12.315 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight setattr -> 25 2021-01-15T19:08:12.315 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight rmattr -> 25 2021-01-15T19:08:12.315 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight copy_from -> 50 2021-01-15T19:08:12.315 INFO:tasks.rados.rados.0.smithi001.stdout:adding op weight append -> 100 2021-01-15T19:08:12.316 INFO:tasks.rados.rados.0.smithi001.stdout:ceph version 16.0.0-9031-g892c51dd (892c51dd3c5f7108e766bea30cd5e0d801b0abd3) pacific (dev) 2021-01-15T19:08:12.316 INFO:tasks.rados.rados.0.smithi001.stdout:Configuration: 2021-01-15T19:08:12.316 INFO:tasks.rados.rados.0.smithi001.stdout: Number of operations: 4000 2021-01-15T19:08:12.316 INFO:tasks.rados.rados.0.smithi001.stdout: Number of objects: 50 2021-01-15T19:08:12.316 INFO:tasks.rados.rados.0.smithi001.stdout: Max in flight operations: 16 2021-01-15T19:08:12.317 INFO:tasks.rados.rados.0.smithi001.stdout: Object size (in bytes): 4000000 2021-01-15T19:08:12.317 INFO:tasks.rados.rados.0.smithi001.stdout: Write stride min: 400000 2021-01-15T19:08:12.317 INFO:tasks.rados.rados.0.smithi001.stdout: Write stride max: 800000 2021-01-15T19:08:12.320 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:12.318+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:12.369 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.369 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:12.380 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:12.425 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:12.423+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:12.526 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:12.524+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:12.558 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.558 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:12.568 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:12.584 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:12.595 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2021-01-15T19:08:12.227493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40324,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":38528,"kb_avail":754934396,"statfs":{"total":773094113280,"available":773052821504,"internally_reserved":0,"allocated":1773568,"data_stored":649265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39452672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016381"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.705141+0000","last_change":"2021-01-15T19:08:10.705141+0000","last_active":"2021-01-15T19:08:10.705141+0000","last_peered":"2021-01-15T19:08:10.705141+0000","last_clean":"2021-01-15T19:08:10.705141+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:10.705141+0000","last_undegraded":"2021-01-15T19:08:10.705141+0000","last_fullsized":"2021-01-15T19:08:10.705141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.234384+0000","last_change":"2021-01-15T19:08:10.234384+0000","last_active":"2021-01-15T19:08:10.234384+0000","last_peered":"2021-01-15T19:08:10.234384+0000","last_clean":"2021-01-15T19:08:10.234384+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:10.234384+0000","last_undegraded":"2021-01-15T19:08:10.234384+0000","last_fullsized":"2021-01-15T19:08:10.234384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.704637+0000","last_change":"2021-01-15T19:08:10.704637+0000","last_active":"2021-01-15T19:08:10.704637+0000","last_peered":"2021-01-15T19:08:10.704637+0000","last_clean":"2021-01-15T19:08:10.704637+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:10.704637+0000","last_undegraded":"2021-01-15T19:08:10.704637+0000","last_fullsized":"2021-01-15T19:08:10.704637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.230760+0000","last_change":"2021-01-15T19:08:10.230760+0000","last_active":"2021-01-15T19:08:10.230760+0000","last_peered":"2021-01-15T19:08:10.230760+0000","last_clean":"2021-01-15T19:08:10.230760+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:10.230760+0000","last_undegraded":"2021-01-15T19:08:10.230760+0000","last_fullsized":"2021-01-15T19:08:10.230760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.233765+0000","last_change":"2021-01-15T19:08:10.232421+0000","last_active":"2021-01-15T19:08:10.233765+0000","last_peered":"2021-01-15T19:08:10.233765+0000","last_clean":"2021-01-15T19:08:10.233765+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:10.233765+0000","last_undegraded":"2021-01-15T19:08:10.233765+0000","last_fullsized":"2021-01-15T19:08:10.233765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.704582+0000","last_change":"2021-01-15T19:08:10.704582+0000","last_active":"2021-01-15T19:08:10.704582+0000","last_peered":"2021-01-15T19:08:10.704582+0000","last_clean":"2021-01-15T19:08:10.704582+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:10.704582+0000","last_undegraded":"2021-01-15T19:08:10.704582+0000","last_fullsized":"2021-01-15T19:08:10.704582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.231806+0000","last_change":"2021-01-15T19:08:10.231806+0000","last_active":"2021-01-15T19:08:10.231806+0000","last_peered":"2021-01-15T19:08:10.231806+0000","last_clean":"2021-01-15T19:08:10.231806+0000","last_became_active":"2021-01-15T19:08:10.230943+0000","last_became_peered":"2021-01-15T19:08:10.230943+0000","last_unstale":"2021-01-15T19:08:10.231806+0000","last_undegraded":"2021-01-15T19:08:10.231806+0000","last_fullsized":"2021-01-15T19:08:10.231806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.231854+0000","last_change":"2021-01-15T19:08:10.231854+0000","last_active":"2021-01-15T19:08:10.231854+0000","last_peered":"2021-01-15T19:08:10.231854+0000","last_clean":"2021-01-15T19:08:10.231854+0000","last_became_active":"2021-01-15T19:08:10.231245+0000","last_became_peered":"2021-01-15T19:08:10.231245+0000","last_unstale":"2021-01-15T19:08:10.231854+0000","last_undegraded":"2021-01-15T19:08:10.231854+0000","last_fullsized":"2021-01-15T19:08:10.231854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.224016+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:08:09.224016+0000","last_peered":"2021-01-15T19:08:09.224016+0000","last_clean":"2021-01-15T19:08:09.224016+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:08:09.224016+0000","last_undegraded":"2021-01-15T19:08:09.224016+0000","last_fullsized":"2021-01-15T19:08:09.224016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.231323+0000","last_change":"2021-01-15T19:08:10.231323+0000","last_active":"2021-01-15T19:08:10.231323+0000","last_peered":"2021-01-15T19:08:10.231323+0000","last_clean":"2021-01-15T19:08:10.231323+0000","last_became_active":"2021-01-15T19:08:10.230610+0000","last_became_peered":"2021-01-15T19:08:10.230610+0000","last_unstale":"2021-01-15T19:08:10.231323+0000","last_undegraded":"2021-01-15T19:08:10.231323+0000","last_fullsized":"2021-01-15T19:08:10.231323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.223762+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:08:09.223762+0000","last_peered":"2021-01-15T19:08:09.223762+0000","last_clean":"2021-01-15T19:08:09.223762+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:08:09.223762+0000","last_undegraded":"2021-01-15T19:08:09.223762+0000","last_fullsized":"2021-01-15T19:08:09.223762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.238302+0000","last_change":"2021-01-15T19:08:10.238302+0000","last_active":"2021-01-15T19:08:10.238302+0000","last_peered":"2021-01-15T19:08:10.238302+0000","last_clean":"2021-01-15T19:08:10.238302+0000","last_became_active":"2021-01-15T19:08:10.237812+0000","last_became_peered":"2021-01-15T19:08:10.237812+0000","last_unstale":"2021-01-15T19:08:10.238302+0000","last_undegraded":"2021-01-15T19:08:10.238302+0000","last_fullsized":"2021-01-15T19:08:10.238302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"14","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.221902+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:08:10.221902+0000","last_peered":"2021-01-15T19:08:10.221902+0000","last_clean":"2021-01-15T19:08:10.221902+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:08:10.221902+0000","last_undegraded":"2021-01-15T19:08:10.221902+0000","last_fullsized":"2021-01-15T19:08:10.221902+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.231670+0000","last_change":"2021-01-15T19:08:10.231670+0000","last_active":"2021-01-15T19:08:10.231670+0000","last_peered":"2021-01-15T19:08:10.231670+0000","last_clean":"2021-01-15T19:08:10.231670+0000","last_became_active":"2021-01-15T19:08:10.231211+0000","last_became_peered":"2021-01-15T19:08:10.231211+0000","last_unstale":"2021-01-15T19:08:10.231670+0000","last_undegraded":"2021-01-15T19:08:10.231670+0000","last_fullsized":"2021-01-15T19:08:10.231670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.223556+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:08:09.223556+0000","last_peered":"2021-01-15T19:08:09.223556+0000","last_clean":"2021-01-15T19:08:09.223556+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:08:09.223556+0000","last_undegraded":"2021-01-15T19:08:09.223556+0000","last_fullsized":"2021-01-15T19:08:09.223556+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"17","reported_epoch":"20","state":"active+clean","last_fresh":"2021-01-15T19:08:08.212486+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:08.212486+0000","last_peered":"2021-01-15T19:08:08.212486+0000","last_clean":"2021-01-15T19:08:08.212486+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:08.212486+0000","last_undegraded":"2021-01-15T19:08:08.212486+0000","last_fullsized":"2021-01-15T19:08:08.212486+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.234902+0000","last_change":"2021-01-15T19:08:10.234902+0000","last_active":"2021-01-15T19:08:10.234902+0000","last_peered":"2021-01-15T19:08:10.234902+0000","last_clean":"2021-01-15T19:08:10.234902+0000","last_became_active":"2021-01-15T19:08:10.234192+0000","last_became_peered":"2021-01-15T19:08:10.234192+0000","last_unstale":"2021-01-15T19:08:10.234902+0000","last_undegraded":"2021-01-15T19:08:10.234902+0000","last_fullsized":"2021-01-15T19:08:10.234902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.223300+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:08:09.223300+0000","last_peered":"2021-01-15T19:08:09.223300+0000","last_clean":"2021-01-15T19:08:09.223300+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:08:09.223300+0000","last_undegraded":"2021-01-15T19:08:09.223300+0000","last_fullsized":"2021-01-15T19:08:09.223300+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.230992+0000","last_change":"2021-01-15T19:08:10.230992+0000","last_active":"2021-01-15T19:08:10.230992+0000","last_peered":"2021-01-15T19:08:10.230992+0000","last_clean":"2021-01-15T19:08:10.230992+0000","last_became_active":"2021-01-15T19:08:10.230297+0000","last_became_peered":"2021-01-15T19:08:10.230297+0000","last_unstale":"2021-01-15T19:08:10.230992+0000","last_undegraded":"2021-01-15T19:08:10.230992+0000","last_fullsized":"2021-01-15T19:08:10.230992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.223927+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:08:09.223927+0000","last_peered":"2021-01-15T19:08:09.223927+0000","last_clean":"2021-01-15T19:08:09.223927+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:08:09.223927+0000","last_undegraded":"2021-01-15T19:08:09.223927+0000","last_fullsized":"2021-01-15T19:08:09.223927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2021-01-15T19:08:10.228930+0000","last_change":"2021-01-15T19:08:10.228930+0000","last_active":"2021-01-15T19:08:10.228930+0000","last_peered":"2021-01-15T19:08:10.228930+0000","last_clean":"2021-01-15T19:08:10.228930+0000","last_became_active":"2021-01-15T19:08:10.228436+0000","last_became_peered":"2021-01-15T19:08:10.228436+0000","last_unstale":"2021-01-15T19:08:10.228930+0000","last_undegraded":"2021-01-15T19:08:10.228930+0000","last_fullsized":"2021-01-15T19:08:10.228930+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":"3","reported_epoch":"22","state":"creating+activating","last_fresh":"2021-01-15T19:08:10.221869+0000","last_change":"2021-01-15T19:08:10.221869+0000","last_active":"2021-01-15T19:08:09.211024+0000","last_peered":"2021-01-15T19:08:09.211024+0000","last_clean":"2021-01-15T19:08:09.211024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:10.221869+0000","last_undegraded":"2021-01-15T19:08:10.221869+0000","last_fullsized":"2021-01-15T19:08:10.221869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":["4(0)","1(2)","6(1)"],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.226124+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:08:09.226124+0000","last_peered":"2021-01-15T19:08:09.226124+0000","last_clean":"2021-01-15T19:08:09.226124+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:08:09.226124+0000","last_undegraded":"2021-01-15T19:08:09.226124+0000","last_fullsized":"2021-01-15T19:08:09.226124+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-01-15T19:08:09.224059+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:08:09.224059+0000","last_peered":"2021-01-15T19:08:09.224059+0000","last_clean":"2021-01-15T19:08:09.224059+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:08:09.224059+0000","last_undegraded":"2021-01-15T19:08:09.224059+0000","last_fullsized":"2021-01-15T19:08:09.224059+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":"3","reported_epoch":"22","state":"creating+activating","last_fresh":"2021-01-15T19:08:10.220914+0000","last_change":"2021-01-15T19:08:10.220914+0000","last_active":"2021-01-15T19:08:09.211024+0000","last_peered":"2021-01-15T19:08:09.211024+0000","last_clean":"2021-01-15T19:08:09.211024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:10.220914+0000","last_undegraded":"2021-01-15T19:08:10.220914+0000","last_fullsized":"2021-01-15T19:08:10.220914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":["4(0)","1(2)","7(1)"],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5096,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366744,"statfs":{"total":96636764160,"available":96631545856,"internally_reserved":0,"allocated":229376,"data_stored":85245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5020,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366820,"statfs":{"total":96636764160,"available":96631623680,"internally_reserved":0,"allocated":217088,"data_stored":78706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5020,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366820,"statfs":{"total":96636764160,"available":96631623680,"internally_reserved":0,"allocated":217088,"data_stored":78706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5032,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366808,"statfs":{"total":96636764160,"available":96631611392,"internally_reserved":0,"allocated":229376,"data_stored":85245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.353}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5020,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366820,"statfs":{"total":96636764160,"available":96631623680,"internally_reserved":0,"allocated":217088,"data_stored":78706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5096,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366744,"statfs":{"total":96636764160,"available":96631545856,"internally_reserved":0,"allocated":229376,"data_stored":85245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5020,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366820,"statfs":{"total":96636764160,"available":96631623680,"internally_reserved":0,"allocated":217088,"data_stored":78706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.71}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0329999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5020,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366820,"statfs":{"total":96636764160,"available":96631623680,"internally_reserved":0,"allocated":217088,"data_stored":78706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]}]}]}} 2021-01-15T19:08:12.627 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:12.625+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:12.715 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.716 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:12.725 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:12.727 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:12.725+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:12.828 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:12.827+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:12.857 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.857 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:12.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:12.930 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:12.928+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:12.998 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:12.999 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:13.008 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:13.053 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:13.051+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:13.131 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:13.130+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:13.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:13.146 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:13.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:13.234 INFO:tasks.rados.rados.0.smithi001.stdout:1: write initial oid 1 2021-01-15T19:08:13.235 INFO:tasks.rados.rados.0.smithi001.stdout:1: seq_num 0 ranges {0=24576} 2021-01-15T19:08:13.235 INFO:tasks.rados.rados.0.smithi001.stdout:1: writing smithi00140323-1 from 0 to 24576 tid 1 2021-01-15T19:08:13.235 INFO:tasks.rados.rados.0.smithi001.stdout:2: write initial oid 2 2021-01-15T19:08:13.236 INFO:tasks.rados.rados.0.smithi001.stdout:2: seq_num 1 ranges {0=24576} 2021-01-15T19:08:13.236 INFO:tasks.rados.rados.0.smithi001.stdout:2: writing smithi00140323-2 from 0 to 24576 tid 1 2021-01-15T19:08:13.236 INFO:tasks.rados.rados.0.smithi001.stdout:3: write initial oid 3 2021-01-15T19:08:13.236 INFO:tasks.rados.rados.0.smithi001.stdout:3: seq_num 2 ranges {0=40960} 2021-01-15T19:08:13.237 INFO:tasks.rados.rados.0.smithi001.stdout:3: writing smithi00140323-3 from 0 to 40960 tid 1 2021-01-15T19:08:13.237 INFO:tasks.rados.rados.0.smithi001.stdout:4: write initial oid 4 2021-01-15T19:08:13.237 INFO:tasks.rados.rados.0.smithi001.stdout:4: seq_num 3 ranges {0=57344} 2021-01-15T19:08:13.238 INFO:tasks.rados.rados.0.smithi001.stdout:4: writing smithi00140323-4 from 0 to 57344 tid 1 2021-01-15T19:08:13.239 INFO:tasks.rados.rados.0.smithi001.stdout:5: write initial oid 5 2021-01-15T19:08:13.239 INFO:tasks.rados.rados.0.smithi001.stdout:5: seq_num 4 ranges {0=73728} 2021-01-15T19:08:13.239 INFO:tasks.rados.rados.0.smithi001.stdout:5: writing smithi00140323-5 from 0 to 73728 tid 1 2021-01-15T19:08:13.240 INFO:tasks.rados.rados.0.smithi001.stdout:6: write initial oid 6 2021-01-15T19:08:13.240 INFO:tasks.rados.rados.0.smithi001.stdout:6: seq_num 5 ranges {0=90112} 2021-01-15T19:08:13.240 INFO:tasks.rados.rados.0.smithi001.stdout:6: writing smithi00140323-6 from 0 to 90112 tid 1 2021-01-15T19:08:13.240 INFO:tasks.rados.rados.0.smithi001.stdout:7: write initial oid 7 2021-01-15T19:08:13.241 INFO:tasks.rados.rados.0.smithi001.stdout:7: seq_num 6 ranges {0=106496} 2021-01-15T19:08:13.241 INFO:tasks.rados.rados.0.smithi001.stdout:7: writing smithi00140323-7 from 0 to 106496 tid 1 2021-01-15T19:08:13.241 INFO:tasks.rados.rados.0.smithi001.stdout:8: write initial oid 8 2021-01-15T19:08:13.241 INFO:tasks.rados.rados.0.smithi001.stdout:8: seq_num 7 ranges {0=122880} 2021-01-15T19:08:13.242 INFO:tasks.rados.rados.0.smithi001.stdout:8: writing smithi00140323-8 from 0 to 122880 tid 1 2021-01-15T19:08:13.242 INFO:tasks.rados.rados.0.smithi001.stdout:9: write initial oid 9 2021-01-15T19:08:13.242 INFO:tasks.rados.rados.0.smithi001.stdout:9: seq_num 8 ranges {0=139264} 2021-01-15T19:08:13.243 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:13.239+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:13.243 INFO:tasks.rados.rados.0.smithi001.stdout:9: writing smithi00140323-9 from 0 to 139264 tid 1 2021-01-15T19:08:13.244 INFO:tasks.rados.rados.0.smithi001.stdout:10: write initial oid 10 2021-01-15T19:08:13.244 INFO:tasks.rados.rados.0.smithi001.stdout:10: seq_num 9 ranges {0=155648} 2021-01-15T19:08:13.244 INFO:tasks.rados.rados.0.smithi001.stdout:10: writing smithi00140323-10 from 0 to 155648 tid 1 2021-01-15T19:08:13.244 INFO:tasks.rados.rados.0.smithi001.stdout:11: write initial oid 11 2021-01-15T19:08:13.245 INFO:tasks.rados.rados.0.smithi001.stdout:11: seq_num 10 ranges {0=172032} 2021-01-15T19:08:13.245 INFO:tasks.rados.rados.0.smithi001.stdout:11: writing smithi00140323-11 from 0 to 172032 tid 1 2021-01-15T19:08:13.245 INFO:tasks.rados.rados.0.smithi001.stdout:12: write initial oid 12 2021-01-15T19:08:13.245 INFO:tasks.rados.rados.0.smithi001.stdout:12: seq_num 11 ranges {0=188416} 2021-01-15T19:08:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:08:13.246 INFO:tasks.rados.rados.0.smithi001.stdout:12: writing smithi00140323-12 from 0 to 188416 tid 1 2021-01-15T19:08:13.247 INFO:tasks.rados.rados.0.smithi001.stdout:13: write initial oid 13 2021-01-15T19:08:13.247 INFO:tasks.rados.rados.0.smithi001.stdout:13: seq_num 12 ranges {0=204800} 2021-01-15T19:08:13.247 INFO:tasks.rados.rados.0.smithi001.stdout:13: writing smithi00140323-13 from 0 to 204800 tid 1 2021-01-15T19:08:13.248 INFO:tasks.rados.rados.0.smithi001.stdout:3: finishing write tid 1 to smithi00140323-3 2021-01-15T19:08:13.249 INFO:tasks.rados.rados.0.smithi001.stdout:4: finishing write tid 1 to smithi00140323-4 2021-01-15T19:08:13.249 INFO:tasks.rados.rados.0.smithi001.stdout:1: finishing write tid 1 to smithi00140323-1 2021-01-15T19:08:13.249 INFO:tasks.rados.rados.0.smithi001.stdout:2: finishing write tid 1 to smithi00140323-2 2021-01-15T19:08:13.249 INFO:tasks.rados.rados.0.smithi001.stdout:14: write initial oid 14 2021-01-15T19:08:13.250 INFO:tasks.rados.rados.0.smithi001.stdout:14: seq_num 13 ranges {0=221184} 2021-01-15T19:08:13.250 INFO:tasks.rados.rados.0.smithi001.stdout:14: writing smithi00140323-14 from 0 to 221184 tid 1 2021-01-15T19:08:13.251 INFO:tasks.rados.rados.0.smithi001.stdout:15: write initial oid 15 2021-01-15T19:08:13.251 INFO:tasks.rados.rados.0.smithi001.stdout:5: finishing write tid 1 to smithi00140323-5 2021-01-15T19:08:13.251 INFO:tasks.rados.rados.0.smithi001.stdout:15: seq_num 14 ranges {0=237568} 2021-01-15T19:08:13.252 INFO:tasks.rados.rados.0.smithi001.stdout:15: writing smithi00140323-15 from 0 to 237568 tid 1 2021-01-15T19:08:13.252 INFO:tasks.rados.rados.0.smithi001.stdout:16: write initial oid 16 2021-01-15T19:08:13.253 INFO:tasks.rados.rados.0.smithi001.stdout:16: seq_num 15 ranges {0=253952} 2021-01-15T19:08:13.254 INFO:tasks.rados.rados.0.smithi001.stdout:16: writing smithi00140323-16 from 0 to 253952 tid 1 2021-01-15T19:08:13.254 INFO:tasks.rados.rados.0.smithi001.stdout:7: finishing write tid 1 to smithi00140323-7 2021-01-15T19:08:13.255 INFO:tasks.rados.rados.0.smithi001.stdout:12: finishing write tid 1 to smithi00140323-12 2021-01-15T19:08:13.255 INFO:tasks.rados.rados.0.smithi001.stdout: waiting on 16 2021-01-15T19:08:13.255 INFO:tasks.rados.rados.0.smithi001.stdout:4: finishing write tid 2 to smithi00140323-4 2021-01-15T19:08:13.256 INFO:tasks.rados.rados.0.smithi001.stdout:2: finishing write tid 2 to smithi00140323-2 2021-01-15T19:08:13.256 INFO:tasks.rados.rados.0.smithi001.stdout:8: finishing write tid 1 to smithi00140323-8 2021-01-15T19:08:13.257 INFO:tasks.rados.rados.0.smithi001.stdout:1: finishing write tid 2 to smithi00140323-1 2021-01-15T19:08:13.258 INFO:tasks.rados.rados.0.smithi001.stdout:1: finishing write tid 3 to smithi00140323-1 2021-01-15T19:08:13.258 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2021-01-15T19:08:13.259 INFO:tasks.rados.rados.0.smithi001.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2021-01-15T19:08:13.259 INFO:tasks.rados.rados.0.smithi001.stdout:11: finishing write tid 1 to smithi00140323-11 2021-01-15T19:08:13.260 INFO:tasks.rados.rados.0.smithi001.stdout:4: finishing write tid 3 to smithi00140323-4 2021-01-15T19:08:13.260 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2021-01-15T19:08:13.260 INFO:tasks.rados.rados.0.smithi001.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2021-01-15T19:08:13.261 INFO:tasks.rados.rados.0.smithi001.stdout:14: finishing write tid 1 to smithi00140323-14 2021-01-15T19:08:13.261 INFO:tasks.rados.rados.0.smithi001.stdout:9: finishing write tid 1 to smithi00140323-9 2021-01-15T19:08:13.261 INFO:tasks.rados.rados.0.smithi001.stdout:1: done (15 left) 2021-01-15T19:08:13.261 INFO:tasks.rados.rados.0.smithi001.stdout:4: done (14 left) 2021-01-15T19:08:13.261 INFO:tasks.rados.rados.0.smithi001.stdout:17: write initial oid 17 2021-01-15T19:08:13.262 INFO:tasks.rados.rados.0.smithi001.stdout:17: seq_num 16 ranges {0=270336} 2021-01-15T19:08:13.263 INFO:tasks.rados.rados.0.smithi001.stdout:17: writing smithi00140323-17 from 0 to 270336 tid 1 2021-01-15T19:08:13.263 INFO:tasks.rados.rados.0.smithi001.stdout:18: write initial oid 18 2021-01-15T19:08:13.263 INFO:tasks.rados.rados.0.smithi001.stdout:18: seq_num 17 ranges {0=286720} 2021-01-15T19:08:13.265 INFO:tasks.rados.rados.0.smithi001.stdout:18: writing smithi00140323-18 from 0 to 286720 tid 1 2021-01-15T19:08:13.265 INFO:tasks.rados.rados.0.smithi001.stdout: waiting on 16 2021-01-15T19:08:13.265 INFO:tasks.rados.rados.0.smithi001.stdout:3: finishing write tid 2 to smithi00140323-3 2021-01-15T19:08:13.265 INFO:tasks.rados.rados.0.smithi001.stdout:6: finishing write tid 1 to smithi00140323-6 2021-01-15T19:08:13.266 INFO:tasks.rados.rados.0.smithi001.stdout:7: finishing write tid 2 to smithi00140323-7 2021-01-15T19:08:13.266 INFO:tasks.rados.rados.0.smithi001.stdout:10: finishing write tid 1 to smithi00140323-10 2021-01-15T19:08:13.266 INFO:tasks.rados.rados.0.smithi001.stdout:13: finishing write tid 1 to smithi00140323-13 2021-01-15T19:08:13.266 INFO:tasks.rados.rados.0.smithi001.stdout:5: finishing write tid 2 to smithi00140323-5 2021-01-15T19:08:13.266 INFO:tasks.rados.rados.0.smithi001.stdout:2: finishing write tid 3 to smithi00140323-2 2021-01-15T19:08:13.267 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2021-01-15T19:08:13.267 INFO:tasks.rados.rados.0.smithi001.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2021-01-15T19:08:13.267 INFO:tasks.rados.rados.0.smithi001.stdout:3: finishing write tid 3 to smithi00140323-3 2021-01-15T19:08:13.267 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2021-01-15T19:08:13.267 INFO:tasks.rados.rados.0.smithi001.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2021-01-15T19:08:13.268 INFO:tasks.rados.rados.0.smithi001.stdout:12: finishing write tid 2 to smithi00140323-12 2021-01-15T19:08:13.268 INFO:tasks.rados.rados.0.smithi001.stdout:7: finishing write tid 3 to smithi00140323-7 2021-01-15T19:08:13.268 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2021-01-15T19:08:13.268 INFO:tasks.rados.rados.0.smithi001.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2021-01-15T19:08:13.268 INFO:tasks.rados.rados.0.smithi001.stdout:5: finishing write tid 3 to smithi00140323-5 2021-01-15T19:08:13.269 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 4 (ObjNum 4 snap 0 seq_num 4) dirty exists 2021-01-15T19:08:13.269 INFO:tasks.rados.rados.0.smithi001.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:08:13.269 INFO:tasks.rados.rados.0.smithi001.stdout:2: done (15 left) 2021-01-15T19:08:13.269 INFO:tasks.rados.rados.0.smithi001.stdout:3: done (14 left) 2021-01-15T19:08:13.269 INFO:tasks.rados.rados.0.smithi001.stdout:5: done (13 left) 2021-01-15T19:08:13.270 INFO:tasks.rados.rados.0.smithi001.stdout:7: done (12 left) 2021-01-15T19:08:13.270 INFO:tasks.rados.rados.0.smithi001.stdout:19: write initial oid 19 2021-01-15T19:08:13.270 INFO:tasks.rados.rados.0.smithi001.stdout:19: seq_num 18 ranges {0=303104} 2021-01-15T19:08:13.271 INFO:tasks.rados.rados.0.smithi001.stdout:19: writing smithi00140323-19 from 0 to 303104 tid 1 2021-01-15T19:08:13.271 INFO:tasks.rados.rados.0.smithi001.stdout:15: finishing write tid 1 to smithi00140323-15 2021-01-15T19:08:13.272 INFO:tasks.rados.rados.0.smithi001.stdout:12: finishing write tid 3 to smithi00140323-12 2021-01-15T19:08:13.272 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2021-01-15T19:08:13.272 INFO:tasks.rados.rados.0.smithi001.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2021-01-15T19:08:13.272 INFO:tasks.rados.rados.0.smithi001.stdout:11: finishing write tid 2 to smithi00140323-11 2021-01-15T19:08:13.272 INFO:tasks.rados.rados.0.smithi001.stdout:16: finishing write tid 1 to smithi00140323-16 2021-01-15T19:08:13.273 INFO:tasks.rados.rados.0.smithi001.stdout:12: done (12 left) 2021-01-15T19:08:13.273 INFO:tasks.rados.rados.0.smithi001.stdout:20: write initial oid 20 2021-01-15T19:08:13.273 INFO:tasks.rados.rados.0.smithi001.stdout:20: seq_num 19 ranges {0=319488} 2021-01-15T19:08:13.273 INFO:tasks.rados.rados.0.smithi001.stdout:20: writing smithi00140323-20 from 0 to 319488 tid 1 2021-01-15T19:08:13.273 INFO:tasks.rados.rados.0.smithi001.stdout:21: write initial oid 21 2021-01-15T19:08:13.274 INFO:tasks.rados.rados.0.smithi001.stdout:21: seq_num 20 ranges {0=344064} 2021-01-15T19:08:13.275 INFO:tasks.rados.rados.0.smithi001.stdout:21: writing smithi00140323-21 from 0 to 344064 tid 1 2021-01-15T19:08:13.275 INFO:tasks.rados.rados.0.smithi001.stdout:11: finishing write tid 3 to smithi00140323-11 2021-01-15T19:08:13.276 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2021-01-15T19:08:13.276 INFO:tasks.rados.rados.0.smithi001.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2021-01-15T19:08:13.276 INFO:tasks.rados.rados.0.smithi001.stdout:9: finishing write tid 2 to smithi00140323-9 2021-01-15T19:08:13.276 INFO:tasks.rados.rados.0.smithi001.stdout:17: finishing write tid 1 to smithi00140323-17 2021-01-15T19:08:13.277 INFO:tasks.rados.rados.0.smithi001.stdout:8: finishing write tid 2 to smithi00140323-8 2021-01-15T19:08:13.277 INFO:tasks.rados.rados.0.smithi001.stdout:6: finishing write tid 2 to smithi00140323-6 2021-01-15T19:08:13.277 INFO:tasks.rados.rados.0.smithi001.stdout:13: finishing write tid 2 to smithi00140323-13 2021-01-15T19:08:13.277 INFO:tasks.rados.rados.0.smithi001.stdout:10: finishing write tid 2 to smithi00140323-10 2021-01-15T19:08:13.277 INFO:tasks.rados.rados.0.smithi001.stdout:14: finishing write tid 2 to smithi00140323-14 2021-01-15T19:08:13.278 INFO:tasks.rados.rados.0.smithi001.stdout:14: finishing write tid 3 to smithi00140323-14 2021-01-15T19:08:13.278 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 2 (ObjNum 13 snap 0 seq_num 13) dirty exists 2021-01-15T19:08:13.278 INFO:tasks.rados.rados.0.smithi001.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2021-01-15T19:08:13.278 INFO:tasks.rados.rados.0.smithi001.stdout:6: finishing write tid 3 to smithi00140323-6 2021-01-15T19:08:13.278 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 6 v 5 (ObjNum 5 snap 0 seq_num 5) dirty exists 2021-01-15T19:08:13.279 INFO:tasks.rados.rados.0.smithi001.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2021-01-15T19:08:13.279 INFO:tasks.rados.rados.0.smithi001.stdout:6: done (13 left) 2021-01-15T19:08:13.279 INFO:tasks.rados.rados.0.smithi001.stdout:11: done (12 left) 2021-01-15T19:08:13.279 INFO:tasks.rados.rados.0.smithi001.stdout:14: done (11 left) 2021-01-15T19:08:13.279 INFO:tasks.rados.rados.0.smithi001.stdout:22: write initial oid 22 2021-01-15T19:08:13.280 INFO:tasks.rados.rados.0.smithi001.stdout:22: seq_num 21 ranges {0=360448} 2021-01-15T19:08:13.280 INFO:tasks.rados.rados.0.smithi001.stdout:22: writing smithi00140323-22 from 0 to 360448 tid 1 2021-01-15T19:08:13.280 INFO:tasks.rados.rados.0.smithi001.stdout:9: finishing write tid 3 to smithi00140323-9 2021-01-15T19:08:13.280 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 9 v 3 (ObjNum 8 snap 0 seq_num 8) dirty exists 2021-01-15T19:08:13.280 INFO:tasks.rados.rados.0.smithi001.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2021-01-15T19:08:13.281 INFO:tasks.rados.rados.0.smithi001.stdout:13: finishing write tid 3 to smithi00140323-13 2021-01-15T19:08:13.281 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2021-01-15T19:08:13.281 INFO:tasks.rados.rados.0.smithi001.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2021-01-15T19:08:13.281 INFO:tasks.rados.rados.0.smithi001.stdout:10: finishing write tid 3 to smithi00140323-10 2021-01-15T19:08:13.281 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2021-01-15T19:08:13.282 INFO:tasks.rados.rados.0.smithi001.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2021-01-15T19:08:13.282 INFO:tasks.rados.rados.0.smithi001.stdout:8: finishing write tid 3 to smithi00140323-8 2021-01-15T19:08:13.282 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2021-01-15T19:08:13.282 INFO:tasks.rados.rados.0.smithi001.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2021-01-15T19:08:13.282 INFO:tasks.rados.rados.0.smithi001.stdout:16: finishing write tid 2 to smithi00140323-16 2021-01-15T19:08:13.283 INFO:tasks.rados.rados.0.smithi001.stdout:8: done (11 left) 2021-01-15T19:08:13.283 INFO:tasks.rados.rados.0.smithi001.stdout:9: done (10 left) 2021-01-15T19:08:13.283 INFO:tasks.rados.rados.0.smithi001.stdout:10: done (9 left) 2021-01-15T19:08:13.283 INFO:tasks.rados.rados.0.smithi001.stdout:13: done (8 left) 2021-01-15T19:08:13.283 INFO:tasks.rados.rados.0.smithi001.stdout:23: write initial oid 23 2021-01-15T19:08:13.284 INFO:tasks.rados.rados.0.smithi001.stdout:23: seq_num 22 ranges {0=376832} 2021-01-15T19:08:13.285 INFO:tasks.rados.rados.0.smithi001.stdout:23: writing smithi00140323-23 from 0 to 376832 tid 1 2021-01-15T19:08:13.285 INFO:tasks.rados.rados.0.smithi001.stdout:24: write initial oid 24 2021-01-15T19:08:13.286 INFO:tasks.rados.rados.0.smithi001.stdout:19: finishing write tid 1 to smithi00140323-19 2021-01-15T19:08:13.286 INFO:tasks.rados.rados.0.smithi001.stdout:15: finishing write tid 2 to smithi00140323-15 2021-01-15T19:08:13.286 INFO:tasks.rados.rados.0.smithi001.stdout:17: finishing write tid 2 to smithi00140323-17 2021-01-15T19:08:13.286 INFO:tasks.rados.rados.0.smithi001.stdout:24: seq_num 23 ranges {0=393216} 2021-01-15T19:08:13.286 INFO:tasks.rados.rados.0.smithi001.stdout:24: writing smithi00140323-24 from 0 to 393216 tid 1 2021-01-15T19:08:13.287 INFO:tasks.rados.rados.0.smithi001.stdout:25: write initial oid 25 2021-01-15T19:08:13.287 INFO:tasks.rados.rados.0.smithi001.stdout:15: finishing write tid 3 to smithi00140323-15 2021-01-15T19:08:13.287 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2021-01-15T19:08:13.287 INFO:tasks.rados.rados.0.smithi001.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2021-01-15T19:08:13.288 INFO:tasks.rados.rados.0.smithi001.stdout:17: finishing write tid 3 to smithi00140323-17 2021-01-15T19:08:13.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2021-01-15T19:08:13.288 INFO:tasks.rados.rados.0.smithi001.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2021-01-15T19:08:13.288 INFO:tasks.rados.rados.0.smithi001.stdout:16: finishing write tid 3 to smithi00140323-16 2021-01-15T19:08:13.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2021-01-15T19:08:13.289 INFO:tasks.rados.rados.0.smithi001.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2021-01-15T19:08:13.289 INFO:tasks.rados.rados.0.smithi001.stdout:18: finishing write tid 1 to smithi00140323-18 2021-01-15T19:08:13.289 INFO:tasks.rados.rados.0.smithi001.stdout:20: finishing write tid 1 to smithi00140323-20 2021-01-15T19:08:13.289 INFO:tasks.rados.rados.0.smithi001.stdout:25: seq_num 24 ranges {0=409600} 2021-01-15T19:08:13.289 INFO:tasks.rados.rados.0.smithi001.stdout:25: writing smithi00140323-25 from 0 to 409600 tid 1 2021-01-15T19:08:13.290 INFO:tasks.rados.rados.0.smithi001.stdout:15: done (10 left) 2021-01-15T19:08:13.290 INFO:tasks.rados.rados.0.smithi001.stdout:16: done (9 left) 2021-01-15T19:08:13.290 INFO:tasks.rados.rados.0.smithi001.stdout:17: done (8 left) 2021-01-15T19:08:13.290 INFO:tasks.rados.rados.0.smithi001.stdout:26: write initial oid 26 2021-01-15T19:08:13.290 INFO:tasks.rados.rados.0.smithi001.stdout:26: seq_num 25 ranges {0=425984} 2021-01-15T19:08:13.291 INFO:tasks.rados.rados.0.smithi001.stdout:26: writing smithi00140323-26 from 0 to 425984 tid 1 2021-01-15T19:08:13.291 INFO:tasks.rados.rados.0.smithi001.stdout:21: finishing write tid 1 to smithi00140323-21 2021-01-15T19:08:13.292 INFO:tasks.rados.rados.0.smithi001.stdout:22: finishing write tid 1 to smithi00140323-22 2021-01-15T19:08:13.292 INFO:tasks.rados.rados.0.smithi001.stdout:27: write initial oid 27 2021-01-15T19:08:13.292 INFO:tasks.rados.rados.0.smithi001.stdout:27: seq_num 26 ranges {0=442368} 2021-01-15T19:08:13.292 INFO:tasks.rados.rados.0.smithi001.stdout:27: writing smithi00140323-27 from 0 to 442368 tid 1 2021-01-15T19:08:13.293 INFO:tasks.rados.rados.0.smithi001.stdout:22: finishing write tid 2 to smithi00140323-22 2021-01-15T19:08:13.293 INFO:tasks.rados.rados.0.smithi001.stdout:18: finishing write tid 2 to smithi00140323-18 2021-01-15T19:08:13.293 INFO:tasks.rados.rados.0.smithi001.stdout:23: finishing write tid 1 to smithi00140323-23 2021-01-15T19:08:13.293 INFO:tasks.rados.rados.0.smithi001.stdout:19: finishing write tid 2 to smithi00140323-19 2021-01-15T19:08:13.294 INFO:tasks.rados.rados.0.smithi001.stdout:21: finishing write tid 2 to smithi00140323-21 2021-01-15T19:08:13.294 INFO:tasks.rados.rados.0.smithi001.stdout:28: write initial oid 28 2021-01-15T19:08:13.294 INFO:tasks.rados.rados.0.smithi001.stdout:28: seq_num 27 ranges {0=458752} 2021-01-15T19:08:13.295 INFO:tasks.rados.rados.0.smithi001.stdout:28: writing smithi00140323-28 from 0 to 458752 tid 1 2021-01-15T19:08:13.296 INFO:tasks.rados.rados.0.smithi001.stdout:29: write initial oid 29 2021-01-15T19:08:13.296 INFO:tasks.rados.rados.0.smithi001.stdout:20: finishing write tid 2 to smithi00140323-20 2021-01-15T19:08:13.296 INFO:tasks.rados.rados.0.smithi001.stdout:24: finishing write tid 1 to smithi00140323-24 2021-01-15T19:08:13.296 INFO:tasks.rados.rados.0.smithi001.stdout:19: finishing write tid 3 to smithi00140323-19 2021-01-15T19:08:13.297 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2021-01-15T19:08:13.297 INFO:tasks.rados.rados.0.smithi001.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2021-01-15T19:08:13.297 INFO:tasks.rados.rados.0.smithi001.stdout:21: finishing write tid 3 to smithi00140323-21 2021-01-15T19:08:13.297 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 2 (ObjNum 20 snap 0 seq_num 20) dirty exists 2021-01-15T19:08:13.297 INFO:tasks.rados.rados.0.smithi001.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2021-01-15T19:08:13.298 INFO:tasks.rados.rados.0.smithi001.stdout:25: finishing write tid 1 to smithi00140323-25 2021-01-15T19:08:13.298 INFO:tasks.rados.rados.0.smithi001.stdout:20: finishing write tid 3 to smithi00140323-20 2021-01-15T19:08:13.298 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:08:13.298 INFO:tasks.rados.rados.0.smithi001.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:08:13.298 INFO:tasks.rados.rados.0.smithi001.stdout:29: seq_num 28 ranges {0=475136} 2021-01-15T19:08:13.299 INFO:tasks.rados.rados.0.smithi001.stdout:29: writing smithi00140323-29 from 0 to 475136 tid 1 2021-01-15T19:08:13.299 INFO:tasks.rados.rados.0.smithi001.stdout:19: done (11 left) 2021-01-15T19:08:13.299 INFO:tasks.rados.rados.0.smithi001.stdout:20: done (10 left) 2021-01-15T19:08:13.299 INFO:tasks.rados.rados.0.smithi001.stdout:21: done (9 left) 2021-01-15T19:08:13.299 INFO:tasks.rados.rados.0.smithi001.stdout:30: write initial oid 30 2021-01-15T19:08:13.300 INFO:tasks.rados.rados.0.smithi001.stdout:18: finishing write tid 3 to smithi00140323-18 2021-01-15T19:08:13.300 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 18 v 4 (ObjNum 17 snap 0 seq_num 17) dirty exists 2021-01-15T19:08:13.300 INFO:tasks.rados.rados.0.smithi001.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2021-01-15T19:08:13.300 INFO:tasks.rados.rados.0.smithi001.stdout:22: finishing write tid 3 to smithi00140323-22 2021-01-15T19:08:13.300 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 22 v 8 (ObjNum 21 snap 0 seq_num 21) dirty exists 2021-01-15T19:08:13.301 INFO:tasks.rados.rados.0.smithi001.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2021-01-15T19:08:13.301 INFO:tasks.rados.rados.0.smithi001.stdout:30: seq_num 29 ranges {0=491520} 2021-01-15T19:08:13.302 INFO:tasks.rados.rados.0.smithi001.stdout:30: writing smithi00140323-30 from 0 to 491520 tid 1 2021-01-15T19:08:13.302 INFO:tasks.rados.rados.0.smithi001.stdout:18: done (9 left) 2021-01-15T19:08:13.302 INFO:tasks.rados.rados.0.smithi001.stdout:22: done (8 left) 2021-01-15T19:08:13.302 INFO:tasks.rados.rados.0.smithi001.stdout:31: write initial oid 31 2021-01-15T19:08:13.302 INFO:tasks.rados.rados.0.smithi001.stdout:31: seq_num 30 ranges {0=507904} 2021-01-15T19:08:13.303 INFO:tasks.rados.rados.0.smithi001.stdout:31: writing smithi00140323-31 from 0 to 507904 tid 1 2021-01-15T19:08:13.303 INFO:tasks.rados.rados.0.smithi001.stdout:32: write initial oid 32 2021-01-15T19:08:13.304 INFO:tasks.rados.rados.0.smithi001.stdout:32: seq_num 31 ranges {0=524288} 2021-01-15T19:08:13.307 INFO:tasks.rados.rados.0.smithi001.stdout:32: writing smithi00140323-32 from 0 to 524288 tid 1 2021-01-15T19:08:13.307 INFO:tasks.rados.rados.0.smithi001.stdout:33: write initial oid 33 2021-01-15T19:08:13.308 INFO:tasks.rados.rados.0.smithi001.stdout:33: seq_num 32 ranges {0=540672} 2021-01-15T19:08:13.311 INFO:tasks.rados.rados.0.smithi001.stdout:33: writing smithi00140323-33 from 0 to 540672 tid 1 2021-01-15T19:08:13.311 INFO:tasks.rados.rados.0.smithi001.stdout:34: write initial oid 34 2021-01-15T19:08:13.311 INFO:tasks.rados.rados.0.smithi001.stdout:34: seq_num 33 ranges {0=557056} 2021-01-15T19:08:13.315 INFO:tasks.rados.rados.0.smithi001.stdout:34: writing smithi00140323-34 from 0 to 557056 tid 1 2021-01-15T19:08:13.316 INFO:tasks.rados.rados.0.smithi001.stdout:26: finishing write tid 1 to smithi00140323-26 2021-01-15T19:08:13.316 INFO:tasks.rados.rados.0.smithi001.stdout:23: finishing write tid 2 to smithi00140323-23 2021-01-15T19:08:13.316 INFO:tasks.rados.rados.0.smithi001.stdout:24: finishing write tid 2 to smithi00140323-24 2021-01-15T19:08:13.316 INFO:tasks.rados.rados.0.smithi001.stdout:27: finishing write tid 1 to smithi00140323-27 2021-01-15T19:08:13.316 INFO:tasks.rados.rados.0.smithi001.stdout:25: finishing write tid 2 to smithi00140323-25 2021-01-15T19:08:13.317 INFO:tasks.rados.rados.0.smithi001.stdout:23: finishing write tid 3 to smithi00140323-23 2021-01-15T19:08:13.317 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2021-01-15T19:08:13.317 INFO:tasks.rados.rados.0.smithi001.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2021-01-15T19:08:13.317 INFO:tasks.rados.rados.0.smithi001.stdout:24: finishing write tid 3 to smithi00140323-24 2021-01-15T19:08:13.317 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 24 v 5 (ObjNum 23 snap 0 seq_num 23) dirty exists 2021-01-15T19:08:13.318 INFO:tasks.rados.rados.0.smithi001.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2021-01-15T19:08:13.318 INFO:tasks.rados.rados.0.smithi001.stdout:25: finishing write tid 3 to smithi00140323-25 2021-01-15T19:08:13.318 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 4 (ObjNum 24 snap 0 seq_num 24) dirty exists 2021-01-15T19:08:13.318 INFO:tasks.rados.rados.0.smithi001.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2021-01-15T19:08:13.319 INFO:tasks.rados.rados.0.smithi001.stdout:26: finishing write tid 2 to smithi00140323-26 2021-01-15T19:08:13.319 INFO:tasks.rados.rados.0.smithi001.stdout:28: finishing write tid 1 to smithi00140323-28 2021-01-15T19:08:13.319 INFO:tasks.rados.rados.0.smithi001.stdout:26: finishing write tid 3 to smithi00140323-26 2021-01-15T19:08:13.319 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2021-01-15T19:08:13.319 INFO:tasks.rados.rados.0.smithi001.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2021-01-15T19:08:13.320 INFO:tasks.rados.rados.0.smithi001.stdout:29: finishing write tid 1 to smithi00140323-29 2021-01-15T19:08:13.320 INFO:tasks.rados.rados.0.smithi001.stdout:27: finishing write tid 2 to smithi00140323-27 2021-01-15T19:08:13.320 INFO:tasks.rados.rados.0.smithi001.stdout:27: finishing write tid 3 to smithi00140323-27 2021-01-15T19:08:13.320 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 27 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2021-01-15T19:08:13.321 INFO:tasks.rados.rados.0.smithi001.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2021-01-15T19:08:13.321 INFO:tasks.rados.rados.0.smithi001.stdout:29: finishing write tid 2 to smithi00140323-29 2021-01-15T19:08:13.321 INFO:tasks.rados.rados.0.smithi001.stdout:30: finishing write tid 1 to smithi00140323-30 2021-01-15T19:08:13.321 INFO:tasks.rados.rados.0.smithi001.stdout:29: finishing write tid 3 to smithi00140323-29 2021-01-15T19:08:13.322 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 2 (ObjNum 28 snap 0 seq_num 28) dirty exists 2021-01-15T19:08:13.322 INFO:tasks.rados.rados.0.smithi001.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2021-01-15T19:08:13.322 INFO:tasks.rados.rados.0.smithi001.stdout:30: finishing write tid 2 to smithi00140323-30 2021-01-15T19:08:13.322 INFO:tasks.rados.rados.0.smithi001.stdout:28: finishing write tid 2 to smithi00140323-28 2021-01-15T19:08:13.323 INFO:tasks.rados.rados.0.smithi001.stdout:30: finishing write tid 3 to smithi00140323-30 2021-01-15T19:08:13.323 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 30 v 2 (ObjNum 29 snap 0 seq_num 29) dirty exists 2021-01-15T19:08:13.323 INFO:tasks.rados.rados.0.smithi001.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2021-01-15T19:08:13.323 INFO:tasks.rados.rados.0.smithi001.stdout:32: finishing write tid 1 to smithi00140323-32 2021-01-15T19:08:13.323 INFO:tasks.rados.rados.0.smithi001.stdout:31: finishing write tid 1 to smithi00140323-31 2021-01-15T19:08:13.324 INFO:tasks.rados.rados.0.smithi001.stdout:23: done (11 left) 2021-01-15T19:08:13.324 INFO:tasks.rados.rados.0.smithi001.stdout:24: done (10 left) 2021-01-15T19:08:13.324 INFO:tasks.rados.rados.0.smithi001.stdout:25: done (9 left) 2021-01-15T19:08:13.324 INFO:tasks.rados.rados.0.smithi001.stdout:26: done (8 left) 2021-01-15T19:08:13.324 INFO:tasks.rados.rados.0.smithi001.stdout:27: done (7 left) 2021-01-15T19:08:13.325 INFO:tasks.rados.rados.0.smithi001.stdout:29: done (6 left) 2021-01-15T19:08:13.325 INFO:tasks.rados.rados.0.smithi001.stdout:30: done (5 left) 2021-01-15T19:08:13.325 INFO:tasks.rados.rados.0.smithi001.stdout:35: write initial oid 35 2021-01-15T19:08:13.325 INFO:tasks.rados.rados.0.smithi001.stdout:35: seq_num 34 ranges {0=573440} 2021-01-15T19:08:13.326 INFO:tasks.rados.rados.0.smithi001.stdout:35: writing smithi00140323-35 from 0 to 573440 tid 1 2021-01-15T19:08:13.327 INFO:tasks.rados.rados.0.smithi001.stdout:36: write initial oid 36 2021-01-15T19:08:13.327 INFO:tasks.rados.rados.0.smithi001.stdout:36: seq_num 35 ranges {0=589824} 2021-01-15T19:08:13.327 INFO:tasks.rados.rados.0.smithi001.stdout:36: writing smithi00140323-36 from 0 to 589824 tid 1 2021-01-15T19:08:13.327 INFO:tasks.rados.rados.0.smithi001.stdout:28: finishing write tid 3 to smithi00140323-28 2021-01-15T19:08:13.328 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2021-01-15T19:08:13.328 INFO:tasks.rados.rados.0.smithi001.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2021-01-15T19:08:13.328 INFO:tasks.rados.rados.0.smithi001.stdout:33: finishing write tid 1 to smithi00140323-33 2021-01-15T19:08:13.328 INFO:tasks.rados.rados.0.smithi001.stdout:32: finishing write tid 2 to smithi00140323-32 2021-01-15T19:08:13.328 INFO:tasks.rados.rados.0.smithi001.stdout:31: finishing write tid 2 to smithi00140323-31 2021-01-15T19:08:13.329 INFO:tasks.rados.rados.0.smithi001.stdout:32: finishing write tid 3 to smithi00140323-32 2021-01-15T19:08:13.329 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 6 (ObjNum 31 snap 0 seq_num 31) dirty exists 2021-01-15T19:08:13.329 INFO:tasks.rados.rados.0.smithi001.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2021-01-15T19:08:13.329 INFO:tasks.rados.rados.0.smithi001.stdout:31: finishing write tid 3 to smithi00140323-31 2021-01-15T19:08:13.329 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 4 (ObjNum 30 snap 0 seq_num 30) dirty exists 2021-01-15T19:08:13.330 INFO:tasks.rados.rados.0.smithi001.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2021-01-15T19:08:13.330 INFO:tasks.rados.rados.0.smithi001.stdout:34: finishing write tid 1 to smithi00140323-34 2021-01-15T19:08:13.330 INFO:tasks.rados.rados.0.smithi001.stdout:35: finishing write tid 1 to smithi00140323-35 2021-01-15T19:08:13.330 INFO:tasks.rados.rados.0.smithi001.stdout:28: done (6 left) 2021-01-15T19:08:13.330 INFO:tasks.rados.rados.0.smithi001.stdout:31: done (5 left) 2021-01-15T19:08:13.331 INFO:tasks.rados.rados.0.smithi001.stdout:32: done (4 left) 2021-01-15T19:08:13.331 INFO:tasks.rados.rados.0.smithi001.stdout:37: write initial oid 37 2021-01-15T19:08:13.331 INFO:tasks.rados.rados.0.smithi001.stdout:37: seq_num 36 ranges {0=606208} 2021-01-15T19:08:13.332 INFO:tasks.rados.rados.0.smithi001.stdout:37: writing smithi00140323-37 from 0 to 606208 tid 1 2021-01-15T19:08:13.332 INFO:tasks.rados.rados.0.smithi001.stdout:33: finishing write tid 2 to smithi00140323-33 2021-01-15T19:08:13.332 INFO:tasks.rados.rados.0.smithi001.stdout:33: finishing write tid 3 to smithi00140323-33 2021-01-15T19:08:13.332 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 7 (ObjNum 32 snap 0 seq_num 32) dirty exists 2021-01-15T19:08:13.333 INFO:tasks.rados.rados.0.smithi001.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2021-01-15T19:08:13.333 INFO:tasks.rados.rados.0.smithi001.stdout:35: finishing write tid 2 to smithi00140323-35 2021-01-15T19:08:13.333 INFO:tasks.rados.rados.0.smithi001.stdout:33: done (4 left) 2021-01-15T19:08:13.333 INFO:tasks.rados.rados.0.smithi001.stdout:38: write initial oid 38 2021-01-15T19:08:13.334 INFO:tasks.rados.rados.0.smithi001.stdout:38: seq_num 37 ranges {0=622592} 2021-01-15T19:08:13.334 INFO:tasks.rados.rados.0.smithi001.stdout:38: writing smithi00140323-38 from 0 to 622592 tid 1 2021-01-15T19:08:13.335 INFO:tasks.rados.rados.0.smithi001.stdout:39: write initial oid 39 2021-01-15T19:08:13.335 INFO:tasks.rados.rados.0.smithi001.stdout:34: finishing write tid 2 to smithi00140323-34 2021-01-15T19:08:13.335 INFO:tasks.rados.rados.0.smithi001.stdout:35: finishing write tid 3 to smithi00140323-35 2021-01-15T19:08:13.335 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 4 (ObjNum 34 snap 0 seq_num 34) dirty exists 2021-01-15T19:08:13.336 INFO:tasks.rados.rados.0.smithi001.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2021-01-15T19:08:13.336 INFO:tasks.rados.rados.0.smithi001.stdout:34: finishing write tid 3 to smithi00140323-34 2021-01-15T19:08:13.336 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 34 v 8 (ObjNum 33 snap 0 seq_num 33) dirty exists 2021-01-15T19:08:13.336 INFO:tasks.rados.rados.0.smithi001.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2021-01-15T19:08:13.336 INFO:tasks.rados.rados.0.smithi001.stdout:36: finishing write tid 1 to smithi00140323-36 2021-01-15T19:08:13.337 INFO:tasks.rados.rados.0.smithi001.stdout:39: seq_num 38 ranges {0=638976} 2021-01-15T19:08:13.337 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:13.333+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:13.338 INFO:tasks.rados.rados.0.smithi001.stdout:39: writing smithi00140323-39 from 0 to 638976 tid 1 2021-01-15T19:08:13.338 INFO:tasks.rados.rados.0.smithi001.stdout:34: done (5 left) 2021-01-15T19:08:13.338 INFO:tasks.rados.rados.0.smithi001.stdout:35: done (4 left) 2021-01-15T19:08:13.339 INFO:tasks.rados.rados.0.smithi001.stdout:40: write initial oid 40 2021-01-15T19:08:13.339 INFO:tasks.rados.rados.0.smithi001.stdout:40: seq_num 39 ranges {0=663552} 2021-01-15T19:08:13.342 INFO:tasks.rados.rados.0.smithi001.stdout:40: writing smithi00140323-40 from 0 to 663552 tid 1 2021-01-15T19:08:13.342 INFO:tasks.rados.rados.0.smithi001.stdout:41: write initial oid 41 2021-01-15T19:08:13.342 INFO:tasks.rados.rados.0.smithi001.stdout:37: finishing write tid 1 to smithi00140323-37 2021-01-15T19:08:13.343 INFO:tasks.rados.rados.0.smithi001.stdout:36: finishing write tid 2 to smithi00140323-36 2021-01-15T19:08:13.343 INFO:tasks.rados.rados.0.smithi001.stdout:36: finishing write tid 3 to smithi00140323-36 2021-01-15T19:08:13.343 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2021-01-15T19:08:13.343 INFO:tasks.rados.rados.0.smithi001.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2021-01-15T19:08:13.344 INFO:tasks.rados.rados.0.smithi001.stdout:37: finishing write tid 2 to smithi00140323-37 2021-01-15T19:08:13.344 INFO:tasks.rados.rados.0.smithi001.stdout:38: finishing write tid 1 to smithi00140323-38 2021-01-15T19:08:13.344 INFO:tasks.rados.rados.0.smithi001.stdout:37: finishing write tid 3 to smithi00140323-37 2021-01-15T19:08:13.344 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2021-01-15T19:08:13.344 INFO:tasks.rados.rados.0.smithi001.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2021-01-15T19:08:13.345 INFO:tasks.rados.rados.0.smithi001.stdout:41: seq_num 40 ranges {0=679936} 2021-01-15T19:08:13.346 INFO:tasks.rados.rados.0.smithi001.stdout:41: writing smithi00140323-41 from 0 to 679936 tid 1 2021-01-15T19:08:13.347 INFO:tasks.rados.rados.0.smithi001.stdout:36: done (5 left) 2021-01-15T19:08:13.347 INFO:tasks.rados.rados.0.smithi001.stdout:37: done (4 left) 2021-01-15T19:08:13.347 INFO:tasks.rados.rados.0.smithi001.stdout:42: write initial oid 42 2021-01-15T19:08:13.347 INFO:tasks.rados.rados.0.smithi001.stdout:42: seq_num 41 ranges {0=696320} 2021-01-15T19:08:13.351 INFO:tasks.rados.rados.0.smithi001.stdout:42: writing smithi00140323-42 from 0 to 696320 tid 1 2021-01-15T19:08:13.351 INFO:tasks.rados.rados.0.smithi001.stdout:43: write initial oid 43 2021-01-15T19:08:13.352 INFO:tasks.rados.rados.0.smithi001.stdout:43: seq_num 42 ranges {0=712704} 2021-01-15T19:08:13.356 INFO:tasks.rados.rados.0.smithi001.stdout:43: writing smithi00140323-43 from 0 to 712704 tid 1 2021-01-15T19:08:13.357 INFO:tasks.rados.rados.0.smithi001.stdout:44: write initial oid 44 2021-01-15T19:08:13.357 INFO:tasks.rados.rados.0.smithi001.stdout:44: seq_num 43 ranges {0=729088} 2021-01-15T19:08:13.361 INFO:tasks.rados.rados.0.smithi001.stdout:44: writing smithi00140323-44 from 0 to 729088 tid 1 2021-01-15T19:08:13.361 INFO:tasks.rados.rados.0.smithi001.stdout:45: write initial oid 45 2021-01-15T19:08:13.361 INFO:tasks.rados.rados.0.smithi001.stdout:45: seq_num 44 ranges {0=745472} 2021-01-15T19:08:13.367 INFO:tasks.rados.rados.0.smithi001.stdout:45: writing smithi00140323-45 from 0 to 745472 tid 1 2021-01-15T19:08:13.368 INFO:tasks.rados.rados.0.smithi001.stdout:46: write initial oid 46 2021-01-15T19:08:13.368 INFO:tasks.rados.rados.0.smithi001.stdout:39: finishing write tid 1 to smithi00140323-39 2021-01-15T19:08:13.369 INFO:tasks.rados.rados.0.smithi001.stdout:38: finishing write tid 2 to smithi00140323-38 2021-01-15T19:08:13.369 INFO:tasks.rados.rados.0.smithi001.stdout:38: finishing write tid 3 to smithi00140323-38 2021-01-15T19:08:13.369 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2021-01-15T19:08:13.370 INFO:tasks.rados.rados.0.smithi001.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:08:13.370 INFO:tasks.rados.rados.0.smithi001.stdout:39: finishing write tid 2 to smithi00140323-39 2021-01-15T19:08:13.371 INFO:tasks.rados.rados.0.smithi001.stdout:39: finishing write tid 3 to smithi00140323-39 2021-01-15T19:08:13.371 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 2 (ObjNum 38 snap 0 seq_num 38) dirty exists 2021-01-15T19:08:13.372 INFO:tasks.rados.rados.0.smithi001.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2021-01-15T19:08:13.372 INFO:tasks.rados.rados.0.smithi001.stdout:40: finishing write tid 1 to smithi00140323-40 2021-01-15T19:08:13.372 INFO:tasks.rados.rados.0.smithi001.stdout:41: finishing write tid 1 to smithi00140323-41 2021-01-15T19:08:13.373 INFO:tasks.rados.rados.0.smithi001.stdout:40: finishing write tid 2 to smithi00140323-40 2021-01-15T19:08:13.373 INFO:tasks.rados.rados.0.smithi001.stdout:41: finishing write tid 2 to smithi00140323-41 2021-01-15T19:08:13.373 INFO:tasks.rados.rados.0.smithi001.stdout:40: finishing write tid 3 to smithi00140323-40 2021-01-15T19:08:13.374 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2021-01-15T19:08:13.374 INFO:tasks.rados.rados.0.smithi001.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2021-01-15T19:08:13.375 INFO:tasks.rados.rados.0.smithi001.stdout:41: finishing write tid 3 to smithi00140323-41 2021-01-15T19:08:13.375 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 41 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2021-01-15T19:08:13.375 INFO:tasks.rados.rados.0.smithi001.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:08:13.376 INFO:tasks.rados.rados.0.smithi001.stdout:42: finishing write tid 1 to smithi00140323-42 2021-01-15T19:08:13.376 INFO:tasks.rados.rados.0.smithi001.stdout:42: finishing write tid 2 to smithi00140323-42 2021-01-15T19:08:13.377 INFO:tasks.rados.rados.0.smithi001.stdout:46: seq_num 45 ranges {0=761856} 2021-01-15T19:08:13.380 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:13.380 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:13.381 INFO:tasks.rados.rados.0.smithi001.stdout:46: writing smithi00140323-46 from 0 to 761856 tid 1 2021-01-15T19:08:13.381 INFO:tasks.rados.rados.0.smithi001.stdout:38: done (8 left) 2021-01-15T19:08:13.382 INFO:tasks.rados.rados.0.smithi001.stdout:39: done (7 left) 2021-01-15T19:08:13.382 INFO:tasks.rados.rados.0.smithi001.stdout:40: done (6 left) 2021-01-15T19:08:13.382 INFO:tasks.rados.rados.0.smithi001.stdout:41: done (5 left) 2021-01-15T19:08:13.383 INFO:tasks.rados.rados.0.smithi001.stdout:47: write initial oid 47 2021-01-15T19:08:13.383 INFO:tasks.rados.rados.0.smithi001.stdout:43: finishing write tid 1 to smithi00140323-43 2021-01-15T19:08:13.384 INFO:tasks.rados.rados.0.smithi001.stdout:42: finishing write tid 3 to smithi00140323-42 2021-01-15T19:08:13.384 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:08:13.384 INFO:tasks.rados.rados.0.smithi001.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2021-01-15T19:08:13.385 INFO:tasks.rados.rados.0.smithi001.stdout:44: finishing write tid 1 to smithi00140323-44 2021-01-15T19:08:13.385 INFO:tasks.rados.rados.0.smithi001.stdout:43: finishing write tid 2 to smithi00140323-43 2021-01-15T19:08:13.386 INFO:tasks.rados.rados.0.smithi001.stdout:44: finishing write tid 2 to smithi00140323-44 2021-01-15T19:08:13.386 INFO:tasks.rados.rados.0.smithi001.stdout:43: finishing write tid 3 to smithi00140323-43 2021-01-15T19:08:13.386 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 8 (ObjNum 42 snap 0 seq_num 42) dirty exists 2021-01-15T19:08:13.387 INFO:tasks.rados.rados.0.smithi001.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2021-01-15T19:08:13.387 INFO:tasks.rados.rados.0.smithi001.stdout:47: seq_num 46 ranges {0=778240} 2021-01-15T19:08:13.388 INFO:tasks.rados.rados.0.smithi001.stdout:47: writing smithi00140323-47 from 0 to 778240 tid 1 2021-01-15T19:08:13.388 INFO:tasks.rados.rados.0.smithi001.stdout:42: done (5 left) 2021-01-15T19:08:13.388 INFO:tasks.rados.rados.0.smithi001.stdout:43: done (4 left) 2021-01-15T19:08:13.389 INFO:tasks.rados.rados.0.smithi001.stdout:48: write initial oid 48 2021-01-15T19:08:13.389 INFO:tasks.rados.rados.0.smithi001.stdout:48: seq_num 47 ranges {0=794624} 2021-01-15T19:08:13.393 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:13.394 INFO:tasks.rados.rados.0.smithi001.stdout:48: writing smithi00140323-48 from 0 to 794624 tid 1 2021-01-15T19:08:13.394 INFO:tasks.rados.rados.0.smithi001.stdout:49: write initial oid 49 2021-01-15T19:08:13.394 INFO:tasks.rados.rados.0.smithi001.stdout:49: seq_num 48 ranges {0=409600,409600=401408} 2021-01-15T19:08:13.395 INFO:tasks.rados.rados.0.smithi001.stdout:49: writing smithi00140323-49 from 0 to 409600 tid 1 2021-01-15T19:08:13.395 INFO:tasks.rados.rados.0.smithi001.stdout:49: writing smithi00140323-49 from 409600 to 811008 tid 2 2021-01-15T19:08:13.396 INFO:tasks.rados.rados.0.smithi001.stdout:50: write initial oid 50 2021-01-15T19:08:13.396 INFO:tasks.rados.rados.0.smithi001.stdout:50: seq_num 49 ranges {0=425984,425984=401408} 2021-01-15T19:08:13.396 INFO:tasks.rados.rados.0.smithi001.stdout:50: writing smithi00140323-50 from 0 to 425984 tid 1 2021-01-15T19:08:13.397 INFO:tasks.rados.rados.0.smithi001.stdout:50: writing smithi00140323-50 from 425984 to 827392 tid 2 2021-01-15T19:08:13.397 INFO:tasks.rados.rados.0.smithi001.stdout:44: finishing write tid 3 to smithi00140323-44 2021-01-15T19:08:13.398 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 44 v 4 (ObjNum 43 snap 0 seq_num 43) dirty exists 2021-01-15T19:08:13.398 INFO:tasks.rados.rados.0.smithi001.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2021-01-15T19:08:13.398 INFO:tasks.rados.rados.0.smithi001.stdout:45: finishing write tid 1 to smithi00140323-45 2021-01-15T19:08:13.399 INFO:tasks.rados.rados.0.smithi001.stdout:45: finishing write tid 2 to smithi00140323-45 2021-01-15T19:08:13.399 INFO:tasks.rados.rados.0.smithi001.stdout:45: finishing write tid 3 to smithi00140323-45 2021-01-15T19:08:13.400 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2021-01-15T19:08:13.400 INFO:tasks.rados.rados.0.smithi001.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2021-01-15T19:08:13.400 INFO:tasks.rados.rados.0.smithi001.stdout:46: finishing write tid 1 to smithi00140323-46 2021-01-15T19:08:13.401 INFO:tasks.rados.rados.0.smithi001.stdout:46: finishing write tid 2 to smithi00140323-46 2021-01-15T19:08:13.401 INFO:tasks.rados.rados.0.smithi001.stdout:46: finishing write tid 3 to smithi00140323-46 2021-01-15T19:08:13.402 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 46 v 10 (ObjNum 45 snap 0 seq_num 45) dirty exists 2021-01-15T19:08:13.402 INFO:tasks.rados.rados.0.smithi001.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2021-01-15T19:08:13.402 INFO:tasks.rados.rados.0.smithi001.stdout:47: finishing write tid 1 to smithi00140323-47 2021-01-15T19:08:13.403 INFO:tasks.rados.rados.0.smithi001.stdout:47: finishing write tid 2 to smithi00140323-47 2021-01-15T19:08:13.403 INFO:tasks.rados.rados.0.smithi001.stdout:47: finishing write tid 3 to smithi00140323-47 2021-01-15T19:08:13.403 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 10 (ObjNum 46 snap 0 seq_num 46) dirty exists 2021-01-15T19:08:13.404 INFO:tasks.rados.rados.0.smithi001.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2021-01-15T19:08:13.404 INFO:tasks.rados.rados.0.smithi001.stdout:48: finishing write tid 1 to smithi00140323-48 2021-01-15T19:08:13.405 INFO:tasks.rados.rados.0.smithi001.stdout:49: finishing write tid 1 to smithi00140323-49 2021-01-15T19:08:13.405 INFO:tasks.rados.rados.0.smithi001.stdout:44: done (6 left) 2021-01-15T19:08:13.405 INFO:tasks.rados.rados.0.smithi001.stdout:45: done (5 left) 2021-01-15T19:08:13.406 INFO:tasks.rados.rados.0.smithi001.stdout:46: done (4 left) 2021-01-15T19:08:13.406 INFO:tasks.rados.rados.0.smithi001.stdout:47: done (3 left) 2021-01-15T19:08:13.407 INFO:tasks.rados.rados.0.smithi001.stdout:51: snap_create 2021-01-15T19:08:13.409 INFO:tasks.rados.rados.0.smithi001.stdout:49: finishing write tid 2 to smithi00140323-49 2021-01-15T19:08:13.409 INFO:tasks.rados.rados.0.smithi001.stdout:49: finishing write tid 3 to smithi00140323-49 2021-01-15T19:08:13.410 INFO:tasks.rados.rados.0.smithi001.stdout:48: finishing write tid 2 to smithi00140323-48 2021-01-15T19:08:13.410 INFO:tasks.rados.rados.0.smithi001.stdout:49: finishing write tid 4 to smithi00140323-49 2021-01-15T19:08:13.411 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2021-01-15T19:08:13.411 INFO:tasks.rados.rados.0.smithi001.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2021-01-15T19:08:13.411 INFO:tasks.rados.rados.0.smithi001.stdout:48: finishing write tid 3 to smithi00140323-48 2021-01-15T19:08:13.412 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 6 (ObjNum 47 snap 0 seq_num 47) dirty exists 2021-01-15T19:08:13.412 INFO:tasks.rados.rados.0.smithi001.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2021-01-15T19:08:13.413 INFO:tasks.rados.rados.0.smithi001.stdout:50: finishing write tid 1 to smithi00140323-50 2021-01-15T19:08:13.413 INFO:tasks.rados.rados.0.smithi001.stdout:50: finishing write tid 2 to smithi00140323-50 2021-01-15T19:08:13.413 INFO:tasks.rados.rados.0.smithi001.stdout:50: finishing write tid 3 to smithi00140323-50 2021-01-15T19:08:13.413 INFO:tasks.rados.rados.0.smithi001.stdout:50: finishing write tid 4 to smithi00140323-50 2021-01-15T19:08:13.414 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 11 (ObjNum 49 snap 0 seq_num 49) dirty exists 2021-01-15T19:08:13.414 INFO:tasks.rados.rados.0.smithi001.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2021-01-15T19:08:13.474 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:13.473+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:13.536 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:13.535+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:13.556 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:13.556 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:13.565 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:13.648 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:13.647+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:13.730 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:13.730 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:13.739 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:13.737+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:13.740 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:13.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:13.840 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:13.838+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:13.904 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:13.905 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:13.914 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:13.940 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:13.938+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:13.969 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:13.990 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:14.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:14.030 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:14.042 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:14.040+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:14.051 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:14.072 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:14.080 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4245.0:2 1.0 e14/13 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head v 14'1, mlcod=0'0)", 2021-01-15T19:08:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:07:51.931673+0000", 2021-01-15T19:08:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.148112615999999, 2021-01-15T19:08:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082302700000000005, 2021-01-15T19:08:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931673+0000", 2021-01-15T19:08:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931673+0000", 2021-01-15T19:08:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4565000000000003e-05 2021-01-15T19:08:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931737+0000", 2021-01-15T19:08:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9440000000000005e-06 2021-01-15T19:08:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931745+0000", 2021-01-15T19:08:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0482999999999999e-05 2021-01-15T19:08:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931756+0000", 2021-01-15T19:08:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1270000000000003e-06 2021-01-15T19:08:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931764+0000", 2021-01-15T19:08:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7147000000000001e-05 2021-01-15T19:08:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931811+0000", 2021-01-15T19:08:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7387000000000001e-05 2021-01-15T19:08:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.931839+0000", 2021-01-15T19:08:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060284300000000004 2021-01-15T19:08:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.932441+0000", 2021-01-15T19:08:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530999999999997e-05 2021-01-15T19:08:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:51.932496+0000", 2021-01-15T19:08:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082302700000000005 2021-01-15T19:08:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4245.0:4 1.0 e14/13 1:916bc198:::INTEL_SSDPEDMX020T4_CVPD514100122P0SGN:head v 14'2, mlcod=0'0)", 2021-01-15T19:08:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:07:52.012390+0000", 2021-01-15T19:08:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.067395087000001, 2021-01-15T19:08:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015440479999999999, 2021-01-15T19:08:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012390+0000", 2021-01-15T19:08:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012390+0000", 2021-01-15T19:08:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1465000000000003e-05 2021-01-15T19:08:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012442+0000", 2021-01-15T19:08:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1226999999999999e-05 2021-01-15T19:08:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012453+0000", 2021-01-15T19:08:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4151000000000001e-05 2021-01-15T19:08:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012467+0000", 2021-01-15T19:08:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4469999999999995e-06 2021-01-15T19:08:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012477+0000", 2021-01-15T19:08:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5005000000000004e-05 2021-01-15T19:08:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012562+0000", 2021-01-15T19:08:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2855000000000001e-05 2021-01-15T19:08:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.012585+0000", 2021-01-15T19:08:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012942889999999999 2021-01-15T19:08:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.013879+0000", 2021-01-15T19:08:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5609000000000002e-05 2021-01-15T19:08:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:07:52.013934+0000", 2021-01-15T19:08:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015440479999999999 2021-01-15T19:08:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.110 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:14.111 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:14.112 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:14.114 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.116 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.116 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.116 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.116 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.117 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.117 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.117 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:14.167 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:14.165+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:14.238 INFO:tasks.rados.rados.0.smithi001.stdout:48: done (3 left) 2021-01-15T19:08:14.239 INFO:tasks.rados.rados.0.smithi001.stdout:49: done (2 left) 2021-01-15T19:08:14.239 INFO:tasks.rados.rados.0.smithi001.stdout:50: done (1 left) 2021-01-15T19:08:14.239 INFO:tasks.rados.rados.0.smithi001.stdout:51: done (0 left) 2021-01-15T19:08:14.239 INFO:tasks.rados.rados.0.smithi001.stdout:52: copy_from oid 1 from oid 48 current snap is 1 2021-01-15T19:08:14.239 INFO:tasks.rados.rados.0.smithi001.stdout:53: rmattr oid 29 current snap is 1 2021-01-15T19:08:14.240 INFO:tasks.rados.rados.0.smithi001.stdout:53: done (1 left) 2021-01-15T19:08:14.240 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 11 current snap is 1 2021-01-15T19:08:14.240 INFO:tasks.rados.rados.0.smithi001.stdout:54: seq_num 50 ranges {172032=442368,614400=401408} 2021-01-15T19:08:14.241 INFO:tasks.rados.rados.0.smithi001.stdout:54: writing smithi00140323-11 from 172032 to 614400 tid 1 2021-01-15T19:08:14.243 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:14.242+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:14.244 INFO:tasks.rados.rados.0.smithi001.stdout:54: writing smithi00140323-11 from 614400 to 1015808 tid 2 2021-01-15T19:08:14.245 INFO:tasks.rados.rados.0.smithi001.stdout:55: read oid 12 snap 0 2021-01-15T19:08:14.245 INFO:tasks.rados.rados.0.smithi001.stdout:55: expect (ObjNum 11 snap 0 seq_num 11) 2021-01-15T19:08:14.246 INFO:tasks.rados.rados.0.smithi001.stdout:56: read oid 38 snap 0 2021-01-15T19:08:14.246 INFO:tasks.rados.rados.0.smithi001.stdout:56: expect (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:08:14.246 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 15 current snap is 1 2021-01-15T19:08:14.246 INFO:tasks.rados.rados.0.smithi001.stdout:57: seq_num 51 ranges {237568=458752,696320=401408} 2021-01-15T19:08:14.249 INFO:tasks.rados.rados.0.smithi001.stdout:57: writing smithi00140323-15 from 237568 to 696320 tid 1 2021-01-15T19:08:14.252 INFO:tasks.rados.rados.0.smithi001.stdout:57: writing smithi00140323-15 from 696320 to 1097728 tid 2 2021-01-15T19:08:14.253 INFO:tasks.rados.rados.0.smithi001.stdout:58: rollback oid 39 current snap is 1 2021-01-15T19:08:14.253 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 39 to 0 2021-01-15T19:08:14.253 INFO:tasks.rados.rados.0.smithi001.stdout:55: expect (ObjNum 11 snap 0 seq_num 11) 2021-01-15T19:08:14.261 INFO:tasks.rados.rados.0.smithi001.stdout:56: expect (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:08:14.278 INFO:tasks.rados.rados.0.smithi001.stdout:54: finishing write tid 1 to smithi00140323-11 2021-01-15T19:08:14.278 INFO:tasks.rados.rados.0.smithi001.stdout:54: finishing write tid 2 to smithi00140323-11 2021-01-15T19:08:14.278 INFO:tasks.rados.rados.0.smithi001.stdout:52: finishing copy_from to smithi00140323-1 2021-01-15T19:08:14.278 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 6 (ObjNum 47 snap 0 seq_num 47) dirty exists 2021-01-15T19:08:14.279 INFO:tasks.rados.rados.0.smithi001.stdout:54: finishing write tid 3 to smithi00140323-11 2021-01-15T19:08:14.279 INFO:tasks.rados.rados.0.smithi001.stdout:52: finishing copy_from racing read to smithi00140323-1 2021-01-15T19:08:14.279 INFO:tasks.rados.rados.0.smithi001.stdout:57: finishing write tid 1 to smithi00140323-15 2021-01-15T19:08:14.279 INFO:tasks.rados.rados.0.smithi001.stdout:52: done (5 left) 2021-01-15T19:08:14.280 INFO:tasks.rados.rados.0.smithi001.stdout:55: done (4 left) 2021-01-15T19:08:14.280 INFO:tasks.rados.rados.0.smithi001.stdout:56: done (3 left) 2021-01-15T19:08:14.280 INFO:tasks.rados.rados.0.smithi001.stdout:59: delete oid 10 current snap is 1 2021-01-15T19:08:14.280 INFO:tasks.rados.rados.0.smithi001.stdout:58: finishing rollback tid 0 to smithi00140323-39 2021-01-15T19:08:14.280 INFO:tasks.rados.rados.0.smithi001.stdout:57: finishing write tid 2 to smithi00140323-15 2021-01-15T19:08:14.281 INFO:tasks.rados.rados.0.smithi001.stdout:57: finishing write tid 3 to smithi00140323-15 2021-01-15T19:08:14.281 INFO:tasks.rados.rados.0.smithi001.stdout:54: finishing write tid 4 to smithi00140323-11 2021-01-15T19:08:14.281 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 14 (ObjNum 50 snap 1 seq_num 50) dirty exists 2021-01-15T19:08:14.281 INFO:tasks.rados.rados.0.smithi001.stdout:54: left oid 11 (ObjNum 50 snap 1 seq_num 50) 2021-01-15T19:08:14.281 INFO:tasks.rados.rados.0.smithi001.stdout:58: finishing rollback tid 1 to smithi00140323-39 2021-01-15T19:08:14.282 INFO:tasks.rados.rados.0.smithi001.stdout:58: finishing rollback tid 2 to smithi00140323-39 2021-01-15T19:08:14.282 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 6 (ObjNum 38 snap 0 seq_num 38) dirty exists 2021-01-15T19:08:14.282 INFO:tasks.rados.rados.0.smithi001.stdout:57: finishing write tid 4 to smithi00140323-15 2021-01-15T19:08:14.282 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 15 v 10 (ObjNum 51 snap 1 seq_num 51) dirty exists 2021-01-15T19:08:14.282 INFO:tasks.rados.rados.0.smithi001.stdout:57: left oid 15 (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:08:14.283 INFO:tasks.rados.rados.0.smithi001.stdout:54: done (3 left) 2021-01-15T19:08:14.284 INFO:tasks.rados.rados.0.smithi001.stdout:57: done (2 left) 2021-01-15T19:08:14.284 INFO:tasks.rados.rados.0.smithi001.stdout:58: done (1 left) 2021-01-15T19:08:14.284 INFO:tasks.rados.rados.0.smithi001.stdout:59: done (0 left) 2021-01-15T19:08:14.284 INFO:tasks.rados.rados.0.smithi001.stdout:60: read oid 16 snap -1 2021-01-15T19:08:14.284 INFO:tasks.rados.rados.0.smithi001.stdout:60: expect (ObjNum 15 snap 0 seq_num 15) 2021-01-15T19:08:14.285 INFO:tasks.rados.rados.0.smithi001.stdout:61: rmattr oid 6 current snap is 1 2021-01-15T19:08:14.285 INFO:tasks.rados.rados.0.smithi001.stdout:61: done (1 left) 2021-01-15T19:08:14.285 INFO:tasks.rados.rados.0.smithi001.stdout:62: setattr oid 26 current snap is 1 2021-01-15T19:08:14.285 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 50 current snap is 1 2021-01-15T19:08:14.285 INFO:tasks.rados.rados.0.smithi001.stdout:63: seq_num 53 ranges {827392=491520,1318912=401408} 2021-01-15T19:08:14.287 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:14.287 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:14.287 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:14.287 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:14.289 INFO:tasks.rados.rados.0.smithi001.stdout:63: writing smithi00140323-50 from 827392 to 1318912 tid 1 2021-01-15T19:08:14.292 INFO:tasks.rados.rados.0.smithi001.stdout:63: writing smithi00140323-50 from 1318912 to 1720320 tid 2 2021-01-15T19:08:14.292 INFO:tasks.rados.rados.0.smithi001.stdout:64: snap_create 2021-01-15T19:08:14.292 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 8 (ObjNum 25 snap 0 seq_num 25) dirty exists 2021-01-15T19:08:14.292 INFO:tasks.rados.rados.0.smithi001.stdout:60: expect (ObjNum 15 snap 0 seq_num 15) 2021-01-15T19:08:14.297 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:14.301 INFO:tasks.rados.rados.0.smithi001.stdout:63: finishing write tid 1 to smithi00140323-50 2021-01-15T19:08:14.301 INFO:tasks.rados.rados.0.smithi001.stdout:63: finishing write tid 2 to smithi00140323-50 2021-01-15T19:08:14.301 INFO:tasks.rados.rados.0.smithi001.stdout:63: finishing write tid 3 to smithi00140323-50 2021-01-15T19:08:14.301 INFO:tasks.rados.rados.0.smithi001.stdout:63: finishing write tid 4 to smithi00140323-50 2021-01-15T19:08:14.302 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 15 (ObjNum 53 snap 1 seq_num 53) dirty exists 2021-01-15T19:08:14.302 INFO:tasks.rados.rados.0.smithi001.stdout:63: left oid 50 (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:08:14.344 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:14.343+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:14.445 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:14.443+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:14.463 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:14.463 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:14.463 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:14.463 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:14.464 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:14.472 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:14.556 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:14.555+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:14.638 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:14.638 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:14.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:14.639 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:14.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=62, reqid=client.4418.0:170, at_version=26'4, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:14.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:14.640 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.38051379699999999, 2021-01-15T19:08:14.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999, 2021-01-15T19:08:14.640 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.640 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.641 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:14.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:14.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530000000000001e-05 2021-01-15T19:08:14.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256020+0000", 2021-01-15T19:08:14.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4555000000000001e-05 2021-01-15T19:08:14.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256044+0000", 2021-01-15T19:08:14.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9589000000000002e-05 2021-01-15T19:08:14.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256064+0000", 2021-01-15T19:08:14.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1316e-05 2021-01-15T19:08:14.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256075+0000", 2021-01-15T19:08:14.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3517999999999996e-05 2021-01-15T19:08:14.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256149+0000", 2021-01-15T19:08:14.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2123999999999999e-05 2021-01-15T19:08:14.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:14.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256161+0000", 2021-01-15T19:08:14.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025493310000000002 2021-01-15T19:08:14.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:14.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258710+0000", 2021-01-15T19:08:14.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0933999999999997e-05 2021-01-15T19:08:14.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258761+0000", 2021-01-15T19:08:14.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999 2021-01-15T19:08:14.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.653 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.654 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=64, reqid=client.4418.0:171, at_version=26'5, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:14.654 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:14.654 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37895316299999998, 2021-01-15T19:08:14.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632711, 2021-01-15T19:08:14.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.655 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.655 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:14.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:14.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7207999999999999e-05 2021-01-15T19:08:14.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257573+0000", 2021-01-15T19:08:14.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2881e-05 2021-01-15T19:08:14.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257596+0000", 2021-01-15T19:08:14.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6541999999999998e-05 2021-01-15T19:08:14.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257613+0000", 2021-01-15T19:08:14.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4219999999999997e-06 2021-01-15T19:08:14.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257619+0000", 2021-01-15T19:08:14.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041151300000000002 2021-01-15T19:08:14.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258030+0000", 2021-01-15T19:08:14.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4518000000000001e-05 2021-01-15T19:08:14.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:14.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258045+0000", 2021-01-15T19:08:14.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001067995 2021-01-15T19:08:14.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:14.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259113+0000", 2021-01-15T19:08:14.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5631999999999999e-05 2021-01-15T19:08:14.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259159+0000", 2021-01-15T19:08:14.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632711 2021-01-15T19:08:14.666 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.667 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.667 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=65, reqid=client.4418.0:172, at_version=26'6, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:14.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:14.668 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37727524600000001, 2021-01-15T19:08:14.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093191900000000002, 2021-01-15T19:08:14.668 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.668 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.669 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:14.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:14.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9619e-05 2021-01-15T19:08:14.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259253+0000", 2021-01-15T19:08:14.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8291000000000002e-05 2021-01-15T19:08:14.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259272+0000", 2021-01-15T19:08:14.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6121e-05 2021-01-15T19:08:14.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259288+0000", 2021-01-15T19:08:14.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6279999999999999e-06 2021-01-15T19:08:14.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259294+0000", 2021-01-15T19:08:14.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4839000000000002e-05 2021-01-15T19:08:14.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259339+0000", 2021-01-15T19:08:14.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0299000000000001e-05 2021-01-15T19:08:14.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:14.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259350+0000", 2021-01-15T19:08:14.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073881299999999995 2021-01-15T19:08:14.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:14.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260088+0000", 2021-01-15T19:08:14.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7308999999999997e-05 2021-01-15T19:08:14.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260136+0000", 2021-01-15T19:08:14.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093191900000000002 2021-01-15T19:08:14.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.679 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=67, reqid=client.4418.0:172, at_version=0'0, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:14.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:14.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.375809112, 2021-01-15T19:08:14.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011872370000000001, 2021-01-15T19:08:14.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:14.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:14.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6808000000000003e-05 2021-01-15T19:08:14.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260717+0000", 2021-01-15T19:08:14.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3665e-05 2021-01-15T19:08:14.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260730+0000", 2021-01-15T19:08:14.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5318000000000001e-05 2021-01-15T19:08:14.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260746+0000", 2021-01-15T19:08:14.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4309999999999999e-06 2021-01-15T19:08:14.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260752+0000", 2021-01-15T19:08:14.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5484e-05 2021-01-15T19:08:14.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260798+0000", 2021-01-15T19:08:14.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4531e-05 2021-01-15T19:08:14.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:14.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260812+0000", 2021-01-15T19:08:14.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00098527100000000002 2021-01-15T19:08:14.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:14.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.261797+0000", 2021-01-15T19:08:14.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9728999999999997e-05 2021-01-15T19:08:14.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.261857+0000", 2021-01-15T19:08:14.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011872370000000001 2021-01-15T19:08:14.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.693 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:177 3.1s0 3:8f2f8304:::smithi00140323-26:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:14.693 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:14.693 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.35493405500000003, 2021-01-15T19:08:14.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494, 2021-01-15T19:08:14.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:14.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:14.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:14.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:14.695 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2021-01-15T19:08:14.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:14.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:14.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6885000000000005e-05 2021-01-15T19:08:14.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281642+0000", 2021-01-15T19:08:14.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:08:14.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281650+0000", 2021-01-15T19:08:14.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2979999999999999e-05 2021-01-15T19:08:14.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281663+0000", 2021-01-15T19:08:14.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1930000000000001e-06 2021-01-15T19:08:14.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281669+0000", 2021-01-15T19:08:14.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880999999999999e-05 2021-01-15T19:08:14.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281724+0000", 2021-01-15T19:08:14.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946600000000001 2021-01-15T19:08:14.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:14.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281853+0000", 2021-01-15T19:08:14.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000582403 2021-01-15T19:08:14.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:14.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282435+0000", 2021-01-15T19:08:14.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013959510000000001 2021-01-15T19:08:14.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:14.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283831+0000", 2021-01-15T19:08:14.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010843630000000001 2021-01-15T19:08:14.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:14.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284916+0000", 2021-01-15T19:08:14.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5765000000000005e-05 2021-01-15T19:08:14.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284981+0000", 2021-01-15T19:08:14.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494 2021-01-15T19:08:14.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.707 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.708 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=27, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:14.708 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:14.709 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.35263082699999998, 2021-01-15T19:08:14.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056741500000000004, 2021-01-15T19:08:14.709 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.709 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.709 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:14.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:14.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5211999999999998e-05 2021-01-15T19:08:14.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283873+0000", 2021-01-15T19:08:14.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2769999999999996e-06 2021-01-15T19:08:14.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283879+0000", 2021-01-15T19:08:14.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0971999999999999e-05 2021-01-15T19:08:14.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283890+0000", 2021-01-15T19:08:14.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1859999999999997e-06 2021-01-15T19:08:14.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283896+0000", 2021-01-15T19:08:14.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050626700000000002 2021-01-15T19:08:14.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284402+0000", 2021-01-15T19:08:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3501e-05 2021-01-15T19:08:14.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284416+0000", 2021-01-15T19:08:14.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056741500000000004 2021-01-15T19:08:14.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=27, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:14.719 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:14.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.351862275, 2021-01-15T19:08:14.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059160400000000002, 2021-01-15T19:08:14.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:14.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7327999999999998e-05 2021-01-15T19:08:14.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284664+0000", 2021-01-15T19:08:14.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8720000000000008e-06 2021-01-15T19:08:14.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284673+0000", 2021-01-15T19:08:14.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6592e-05 2021-01-15T19:08:14.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284690+0000", 2021-01-15T19:08:14.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.725e-06 2021-01-15T19:08:14.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284696+0000", 2021-01-15T19:08:14.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7620000000000006e-05 2021-01-15T19:08:14.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284764+0000", 2021-01-15T19:08:14.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044446699999999998 2021-01-15T19:08:14.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285208+0000", 2021-01-15T19:08:14.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059160400000000002 2021-01-15T19:08:14.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.728 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.729 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=28, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:14.729 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.285722+0000", 2021-01-15T19:08:14.729 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.35075657300000002, 2021-01-15T19:08:14.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110956, 2021-01-15T19:08:14.729 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.730 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.730 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285722+0000", 2021-01-15T19:08:14.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285722+0000", 2021-01-15T19:08:14.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8652999999999997e-05 2021-01-15T19:08:14.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285761+0000", 2021-01-15T19:08:14.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9509999999999997e-06 2021-01-15T19:08:14.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285766+0000", 2021-01-15T19:08:14.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1392e-05 2021-01-15T19:08:14.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285777+0000", 2021-01-15T19:08:14.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4969999999999998e-06 2021-01-15T19:08:14.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285784+0000", 2021-01-15T19:08:14.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1278e-05 2021-01-15T19:08:14.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285825+0000", 2021-01-15T19:08:14.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1850000000000004e-06 2021-01-15T19:08:14.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285833+0000", 2021-01-15T19:08:14.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110956 2021-01-15T19:08:14.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.738 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=28, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.285882+0000", 2021-01-15T19:08:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.350597188, 2021-01-15T19:08:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014946800000000001, 2021-01-15T19:08:14.740 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:14.740 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:14.740 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:14.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:14.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285882+0000", 2021-01-15T19:08:14.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:14.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:14.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285882+0000", 2021-01-15T19:08:14.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1548000000000001e-05 2021-01-15T19:08:14.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:14.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285943+0000", 2021-01-15T19:08:14.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0373999999999999e-05 2021-01-15T19:08:14.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:14.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285954+0000", 2021-01-15T19:08:14.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7356000000000001e-05 2021-01-15T19:08:14.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:14.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285971+0000", 2021-01-15T19:08:14.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9759999999999995e-06 2021-01-15T19:08:14.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:14.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285979+0000", 2021-01-15T19:08:14.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0701999999999998e-05 2021-01-15T19:08:14.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:14.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.286010+0000", 2021-01-15T19:08:14.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1512e-05 2021-01-15T19:08:14.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:14.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:14.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:14.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.286031+0000", 2021-01-15T19:08:14.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014946800000000001 2021-01-15T19:08:14.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:14.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:14.749 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:14.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:14.751 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:14.749+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:14.851 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:14.850+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:14.916 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:14.917 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:14.925 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:14.969 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:14.968+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:15.050 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:15.050 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:15.050 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:15.050 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:15.051 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:15.053 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:15.052+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:15.059 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:15.154 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:15.153+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:15.226 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:15.226 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=34, reqid=client.4418.0:156, at_version=26'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:15.228 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.98126355600000004, 2021-01-15T19:08:15.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002, 2021-01-15T19:08:15.228 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.228 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.228 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:15.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:15.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166446 2021-01-15T19:08:15.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243528+0000", 2021-01-15T19:08:15.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5724e-05 2021-01-15T19:08:15.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243554+0000", 2021-01-15T19:08:15.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3528e-05 2021-01-15T19:08:15.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243567+0000", 2021-01-15T19:08:15.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4610000000000008e-06 2021-01-15T19:08:15.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243576+0000", 2021-01-15T19:08:15.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1644999999999999e-05 2021-01-15T19:08:15.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243618+0000", 2021-01-15T19:08:15.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2930000000000001e-06 2021-01-15T19:08:15.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243626+0000", 2021-01-15T19:08:15.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020585310000000002 2021-01-15T19:08:15.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245684+0000", 2021-01-15T19:08:15.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7432999999999997e-05 2021-01-15T19:08:15.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245742+0000", 2021-01-15T19:08:15.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002 2021-01-15T19:08:15.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.240 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=36, reqid=client.4418.0:157, at_version=26'13, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:15.240 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:15.240 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.97878105500000001, 2021-01-15T19:08:15.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018949259999999999, 2021-01-15T19:08:15.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.241 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:15.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:15.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019949400000000001 2021-01-15T19:08:15.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246044+0000", 2021-01-15T19:08:15.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3130999999999999e-05 2021-01-15T19:08:15.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246067+0000", 2021-01-15T19:08:15.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8967999999999999e-05 2021-01-15T19:08:15.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246086+0000", 2021-01-15T19:08:15.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7729999999999999e-06 2021-01-15T19:08:15.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246094+0000", 2021-01-15T19:08:15.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3356999999999998e-05 2021-01-15T19:08:15.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246137+0000", 2021-01-15T19:08:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0242999999999999e-05 2021-01-15T19:08:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246157+0000", 2021-01-15T19:08:15.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015296000000000001 2021-01-15T19:08:15.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.247687+0000", 2021-01-15T19:08:15.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.236e-05 2021-01-15T19:08:15.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.247739+0000", 2021-01-15T19:08:15.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018949259999999999 2021-01-15T19:08:15.251 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.251 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=37, reqid=client.4418.0:158, at_version=26'14, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:15.252 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.254309+0000", 2021-01-15T19:08:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.97031672899999999, 2021-01-15T19:08:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068320900000000005, 2021-01-15T19:08:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254309+0000", 2021-01-15T19:08:15.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254309+0000", 2021-01-15T19:08:15.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9445999999999999e-05 2021-01-15T19:08:15.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254338+0000", 2021-01-15T19:08:15.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0154000000000001e-05 2021-01-15T19:08:15.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254358+0000", 2021-01-15T19:08:15.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2536e-05 2021-01-15T19:08:15.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:15.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:15.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254378+0000", 2021-01-15T19:08:15.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7002000000000001e-05 2021-01-15T19:08:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254435+0000", 2021-01-15T19:08:15.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2788e-05 2021-01-15T19:08:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254447+0000", 2021-01-15T19:08:15.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049015799999999998 2021-01-15T19:08:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254937+0000", 2021-01-15T19:08:15.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4296000000000003e-05 2021-01-15T19:08:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254992+0000", 2021-01-15T19:08:15.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068320900000000005 2021-01-15T19:08:15.264 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.264 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.264 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.265 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=39, reqid=client.4418.0:158, at_version=0'0, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:15.265 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:15.265 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.96671148600000001, 2021-01-15T19:08:15.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001101858, 2021-01-15T19:08:15.265 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.266 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.266 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:15.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5347999999999999e-05 2021-01-15T19:08:15.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257969+0000", 2021-01-15T19:08:15.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5548000000000001e-05 2021-01-15T19:08:15.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257985+0000", 2021-01-15T19:08:15.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8144000000000001e-05 2021-01-15T19:08:15.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258003+0000", 2021-01-15T19:08:15.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2830000000000004e-06 2021-01-15T19:08:15.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258010+0000", 2021-01-15T19:08:15.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135764 2021-01-15T19:08:15.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258146+0000", 2021-01-15T19:08:15.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3592e-05 2021-01-15T19:08:15.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258159+0000", 2021-01-15T19:08:15.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080877599999999998 2021-01-15T19:08:15.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258968+0000", 2021-01-15T19:08:15.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7403e-05 2021-01-15T19:08:15.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259016+0000", 2021-01-15T19:08:15.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001101858 2021-01-15T19:08:15.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.277 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.cs1 26/21 ECSubRead(tid=40, to_read={3:3ef45b66:::smithi00140323-11:head=0,4096,0}, subchunks={3:3ef45b66:::smithi00140323-11:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.278 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.259308+0000", 2021-01-15T19:08:15.278 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.96531683300000004, 2021-01-15T19:08:15.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041551899999999999, 2021-01-15T19:08:15.278 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.278 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259308+0000", 2021-01-15T19:08:15.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259308+0000", 2021-01-15T19:08:15.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8378999999999999e-05 2021-01-15T19:08:15.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259357+0000", 2021-01-15T19:08:15.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3331e-05 2021-01-15T19:08:15.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259370+0000", 2021-01-15T19:08:15.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8726e-05 2021-01-15T19:08:15.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259389+0000", 2021-01-15T19:08:15.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3130000000000001e-06 2021-01-15T19:08:15.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259396+0000", 2021-01-15T19:08:15.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000198977 2021-01-15T19:08:15.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259595+0000", 2021-01-15T19:08:15.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012879300000000001 2021-01-15T19:08:15.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259724+0000", 2021-01-15T19:08:15.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041551899999999999 2021-01-15T19:08:15.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.287 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.288 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 26/21 ECSubWrite(tid=69, reqid=client.4418.0:173, at_version=26'11, trim_to=0'0, roll_forward_to=25'9))", 2021-01-15T19:08:15.288 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:15.288 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.94601405699999996, 2021-01-15T19:08:15.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087352800000000004, 2021-01-15T19:08:15.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.289 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:15.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:15.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6239000000000002e-05 2021-01-15T19:08:15.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278657+0000", 2021-01-15T19:08:15.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2021-01-15T19:08:15.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278685+0000", 2021-01-15T19:08:15.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8410000000000002e-05 2021-01-15T19:08:15.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278704+0000", 2021-01-15T19:08:15.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550999999999999e-05 2021-01-15T19:08:15.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278714+0000", 2021-01-15T19:08:15.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0232999999999999e-05 2021-01-15T19:08:15.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278754+0000", 2021-01-15T19:08:15.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0690000000000008e-06 2021-01-15T19:08:15.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278764+0000", 2021-01-15T19:08:15.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067577100000000003 2021-01-15T19:08:15.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.279439+0000", 2021-01-15T19:08:15.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5459e-05 2021-01-15T19:08:15.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.279485+0000", 2021-01-15T19:08:15.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087352800000000004 2021-01-15T19:08:15.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 26/21 ECSubWrite(tid=70, reqid=client.4418.0:173, at_version=0'0, trim_to=0'0, roll_forward_to=26'11))", 2021-01-15T19:08:15.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.280242+0000", 2021-01-15T19:08:15.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.94438356899999998, 2021-01-15T19:08:15.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078208300000000004, 2021-01-15T19:08:15.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.301 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280242+0000", 2021-01-15T19:08:15.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280242+0000", 2021-01-15T19:08:15.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:08:15.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280291+0000", 2021-01-15T19:08:15.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4776e-05 2021-01-15T19:08:15.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280306+0000", 2021-01-15T19:08:15.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8844000000000001e-05 2021-01-15T19:08:15.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280325+0000", 2021-01-15T19:08:15.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2719999999999997e-06 2021-01-15T19:08:15.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280333+0000", 2021-01-15T19:08:15.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102852 2021-01-15T19:08:15.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280436+0000", 2021-01-15T19:08:15.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.256e-05 2021-01-15T19:08:15.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280449+0000", 2021-01-15T19:08:15.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052864899999999996 2021-01-15T19:08:15.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280977+0000", 2021-01-15T19:08:15.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6461000000000001e-05 2021-01-15T19:08:15.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281024+0000", 2021-01-15T19:08:15.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078208300000000004 2021-01-15T19:08:15.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=21, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:15.313 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.94226408399999995, 2021-01-15T19:08:15.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010378760000000001, 2021-01-15T19:08:15.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.314 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.314 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:15.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:15.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4665e-05 2021-01-15T19:08:15.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282406+0000", 2021-01-15T19:08:15.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330999999999999e-05 2021-01-15T19:08:15.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282418+0000", 2021-01-15T19:08:15.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8114e-05 2021-01-15T19:08:15.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282436+0000", 2021-01-15T19:08:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4099999999999997e-06 2021-01-15T19:08:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282446+0000", 2021-01-15T19:08:15.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120245 2021-01-15T19:08:15.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282566+0000", 2021-01-15T19:08:15.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083311099999999999 2021-01-15T19:08:15.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283399+0000", 2021-01-15T19:08:15.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010378760000000001 2021-01-15T19:08:15.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.322 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.323 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=22, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.323 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:15.323 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.94191148300000005, 2021-01-15T19:08:15.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090976900000000005, 2021-01-15T19:08:15.324 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.324 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.324 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:15.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:15.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9024999999999999e-05 2021-01-15T19:08:15.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282763+0000", 2021-01-15T19:08:15.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2038999999999999e-05 2021-01-15T19:08:15.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282775+0000", 2021-01-15T19:08:15.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9735999999999999e-05 2021-01-15T19:08:15.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282795+0000", 2021-01-15T19:08:15.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4930000000000003e-06 2021-01-15T19:08:15.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282801+0000", 2021-01-15T19:08:15.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063796299999999996 2021-01-15T19:08:15.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283439+0000", 2021-01-15T19:08:15.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184513 2021-01-15T19:08:15.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283624+0000", 2021-01-15T19:08:15.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090976900000000005 2021-01-15T19:08:15.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.333 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=23, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.333 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.284313+0000", 2021-01-15T19:08:15.334 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.94031271400000005, 2021-01-15T19:08:15.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030638900000000001, 2021-01-15T19:08:15.334 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:15.334 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.335 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:15.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:15.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284313+0000", 2021-01-15T19:08:15.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:15.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:15.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284313+0000", 2021-01-15T19:08:15.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9039999999999998e-05 2021-01-15T19:08:15.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:15.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284362+0000", 2021-01-15T19:08:15.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:08:15.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:15.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284371+0000", 2021-01-15T19:08:15.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0740000000000001e-05 2021-01-15T19:08:15.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:15.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284392+0000", 2021-01-15T19:08:15.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8130000000000002e-06 2021-01-15T19:08:15.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284400+0000", 2021-01-15T19:08:15.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9934e-05 2021-01-15T19:08:15.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:15.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284450+0000", 2021-01-15T19:08:15.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016856 2021-01-15T19:08:15.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:15.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:15.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:15.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284619+0000", 2021-01-15T19:08:15.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030638900000000001 2021-01-15T19:08:15.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:15.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:15.344 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:15.344 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:08:15.347 INFO:tasks.rados.rados.0.smithi001.stdout:60: done (3 left) 2021-01-15T19:08:15.347 INFO:tasks.rados.rados.0.smithi001.stdout:62: done (2 left) 2021-01-15T19:08:15.347 INFO:tasks.rados.rados.0.smithi001.stdout:63: done (1 left) 2021-01-15T19:08:15.347 INFO:tasks.rados.rados.0.smithi001.stdout:64: done (0 left) 2021-01-15T19:08:15.348 INFO:tasks.rados.rados.0.smithi001.stdout:65: rmattr oid 4 current snap is 2 2021-01-15T19:08:15.348 INFO:tasks.rados.rados.0.smithi001.stdout:65: done (0 left) 2021-01-15T19:08:15.348 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 2 2021-01-15T19:08:15.348 INFO:tasks.rados.rados.0.smithi001.stdout:66: seq_num 54 ranges {638976=507904,1146880=401408} 2021-01-15T19:08:15.348 INFO:tasks.rados.rados.0.smithi001.stdout:66: writing smithi00140323-39 from 638976 to 1146880 tid 1 2021-01-15T19:08:15.349 INFO:tasks.rados.rados.0.smithi001.stdout:66: writing smithi00140323-39 from 1146880 to 1548288 tid 2 2021-01-15T19:08:15.349 INFO:tasks.rados.rados.0.smithi001.stdout:67: read oid 15 snap -1 2021-01-15T19:08:15.349 INFO:tasks.rados.rados.0.smithi001.stdout:67: expect (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:08:15.349 INFO:tasks.rados.rados.0.smithi001.stdout:68: rollback oid 43 current snap is 2 2021-01-15T19:08:15.349 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 43 to 1 2021-01-15T19:08:15.350 INFO:tasks.rados.rados.0.smithi001.stdout:69: rmattr oid 22 current snap is 2 2021-01-15T19:08:15.350 INFO:tasks.rados.rados.0.smithi001.stdout:69: done (3 left) 2021-01-15T19:08:15.350 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 2 2021-01-15T19:08:15.350 INFO:tasks.rados.rados.0.smithi001.stdout:70: seq_num 55 ranges {221184=524288,745472=401408} 2021-01-15T19:08:15.350 INFO:tasks.rados.rados.0.smithi001.stdout:70: writing smithi00140323-14 from 221184 to 745472 tid 1 2021-01-15T19:08:15.351 INFO:tasks.rados.rados.0.smithi001.stdout:70: writing smithi00140323-14 from 745472 to 1146880 tid 2 2021-01-15T19:08:15.351 INFO:tasks.rados.rados.0.smithi001.stdout:71: snap_remove snap 0 2021-01-15T19:08:15.351 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:15.344+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:15.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:15.446 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:15.444+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:15.498 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:15.498 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:15.498 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:15.499 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:15.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:15.551 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:15.550+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:15.640 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:15.641 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:15.641 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:15.648 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:15.647+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:15.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:15.749 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:15.748+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:15.823 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:08:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:154 3.fs0 3:f27a3046:::smithi00140323-1:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.582756684, 2021-01-15T19:08:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002, 2021-01-15T19:08:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 154 2021-01-15T19:08:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4352999999999999e-05 2021-01-15T19:08:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237631+0000", 2021-01-15T19:08:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3304e-05 2021-01-15T19:08:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237644+0000", 2021-01-15T19:08:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1736e-05 2021-01-15T19:08:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237666+0000", 2021-01-15T19:08:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0607e-05 2021-01-15T19:08:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237676+0000", 2021-01-15T19:08:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3862e-05 2021-01-15T19:08:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237730+0000", 2021-01-15T19:08:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1849999999999998e-06 2021-01-15T19:08:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237737+0000", 2021-01-15T19:08:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014276250000000001 2021-01-15T19:08:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239165+0000", 2021-01-15T19:08:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7447000000000007e-05 2021-01-15T19:08:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239232+0000", 2021-01-15T19:08:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061349480000000003 2021-01-15T19:08:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245367+0000", 2021-01-15T19:08:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095382000000000002 2021-01-15T19:08:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254906+0000", 2021-01-15T19:08:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113465 2021-01-15T19:08:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255019+0000", 2021-01-15T19:08:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000454659 2021-01-15T19:08:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255474+0000", 2021-01-15T19:08:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002 2021-01-15T19:08:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:155 3.fs0 3:f27a3046:::smithi00140323-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5825955759999999, 2021-01-15T19:08:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999, 2021-01-15T19:08:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 155 2021-01-15T19:08:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9607999999999998e-05 2021-01-15T19:08:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237767+0000", 2021-01-15T19:08:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3810000000000001e-06 2021-01-15T19:08:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237771+0000", 2021-01-15T19:08:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5639999999999999e-06 2021-01-15T19:08:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237779+0000", 2021-01-15T19:08:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.348e-06 2021-01-15T19:08:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237782+0000", 2021-01-15T19:08:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3377000000000002e-05 2021-01-15T19:08:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237816+0000", 2021-01-15T19:08:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000003e-06 2021-01-15T19:08:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237822+0000", 2021-01-15T19:08:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552276 2021-01-15T19:08:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239374+0000", 2021-01-15T19:08:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239415+0000", 2021-01-15T19:08:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010934268 2021-01-15T19:08:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250349+0000", 2021-01-15T19:08:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011671 2021-01-15T19:08:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250466+0000", 2021-01-15T19:08:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050409820000000003 2021-01-15T19:08:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255507+0000", 2021-01-15T19:08:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0531000000000005e-05 2021-01-15T19:08:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255587+0000", 2021-01-15T19:08:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122043 2021-01-15T19:08:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255709+0000", 2021-01-15T19:08:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999 2021-01-15T19:08:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:156 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~442368] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5787552170000001, 2021-01-15T19:08:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759, 2021-01-15T19:08:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156 2021-01-15T19:08:15.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.863 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:15.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:15.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030551200000000003 2021-01-15T19:08:15.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241873+0000", 2021-01-15T19:08:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6080000000000002e-06 2021-01-15T19:08:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241883+0000", 2021-01-15T19:08:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241898+0000", 2021-01-15T19:08:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4010000000000002e-06 2021-01-15T19:08:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241904+0000", 2021-01-15T19:08:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6971999999999997e-05 2021-01-15T19:08:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241951+0000", 2021-01-15T19:08:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104244 2021-01-15T19:08:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242055+0000", 2021-01-15T19:08:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070015099999999999 2021-01-15T19:08:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242755+0000", 2021-01-15T19:08:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0080604460000000006 2021-01-15T19:08:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250816+0000", 2021-01-15T19:08:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00338733 2021-01-15T19:08:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254203+0000", 2021-01-15T19:08:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3444999999999998e-05 2021-01-15T19:08:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254287+0000", 2021-01-15T19:08:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759 2021-01-15T19:08:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:157 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~401408] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5759489900000001, 2021-01-15T19:08:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335, 2021-01-15T19:08:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 157 2021-01-15T19:08:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000301899 2021-01-15T19:08:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244676+0000", 2021-01-15T19:08:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6430000000000001e-06 2021-01-15T19:08:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244685+0000", 2021-01-15T19:08:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5293000000000001e-05 2021-01-15T19:08:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244700+0000", 2021-01-15T19:08:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6499999999999996e-06 2021-01-15T19:08:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244708+0000", 2021-01-15T19:08:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1064999999999997e-05 2021-01-15T19:08:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244789+0000", 2021-01-15T19:08:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117805 2021-01-15T19:08:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244906+0000", 2021-01-15T19:08:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070339100000000004 2021-01-15T19:08:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245610+0000", 2021-01-15T19:08:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087607599999999994 2021-01-15T19:08:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107695 2021-01-15T19:08:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254478+0000", 2021-01-15T19:08:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109134 2021-01-15T19:08:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254587+0000", 2021-01-15T19:08:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335 2021-01-15T19:08:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:158 3.cs0 3:3ef45b66:::smithi00140323-11:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.575562973, 2021-01-15T19:08:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441, 2021-01-15T19:08:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 158 2021-01-15T19:08:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5699999999999999e-05 2021-01-15T19:08:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244816+0000", 2021-01-15T19:08:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6839999999999996e-06 2021-01-15T19:08:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244823+0000", 2021-01-15T19:08:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4626e-05 2021-01-15T19:08:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244838+0000", 2021-01-15T19:08:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6500000000000001e-06 2021-01-15T19:08:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244844+0000", 2021-01-15T19:08:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004205149 2021-01-15T19:08:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249049+0000", 2021-01-15T19:08:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015840599999999999 2021-01-15T19:08:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249207+0000", 2021-01-15T19:08:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071027700000000004 2021-01-15T19:08:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249918+0000", 2021-01-15T19:08:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051411900000000003 2021-01-15T19:08:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255059+0000", 2021-01-15T19:08:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045701699999999999 2021-01-15T19:08:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255516+0000", 2021-01-15T19:08:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8710999999999997e-05 2021-01-15T19:08:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255614+0000", 2021-01-15T19:08:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441 2021-01-15T19:08:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:159 3.cs0 3:3ef45b66:::smithi00140323-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.575430522, 2021-01-15T19:08:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195, 2021-01-15T19:08:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 159 2021-01-15T19:08:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0961000000000003e-05 2021-01-15T19:08:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244933+0000", 2021-01-15T19:08:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0480000000000001e-06 2021-01-15T19:08:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244941+0000", 2021-01-15T19:08:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3711e-05 2021-01-15T19:08:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244954+0000", 2021-01-15T19:08:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2390000000000002e-06 2021-01-15T19:08:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244959+0000", 2021-01-15T19:08:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082968859999999998 2021-01-15T19:08:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253256+0000", 2021-01-15T19:08:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000147608 2021-01-15T19:08:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253404+0000", 2021-01-15T19:08:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047189329999999998 2021-01-15T19:08:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258123+0000", 2021-01-15T19:08:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010107000000000001 2021-01-15T19:08:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258224+0000", 2021-01-15T19:08:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019277389999999999 2021-01-15T19:08:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260152+0000", 2021-01-15T19:08:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195 2021-01-15T19:08:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpRead(3.es1 26/21 ECSubRead(tid=25, to_read={3:72e8704c:::smithi00140323-12:head=0,94208,0}, subchunks={3:72e8704c:::smithi00140323-12:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.245824+0000", 2021-01-15T19:08:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5744987859999999, 2021-01-15T19:08:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031852109999999999, 2021-01-15T19:08:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245824+0000", 2021-01-15T19:08:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245824+0000", 2021-01-15T19:08:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5965999999999997e-05 2021-01-15T19:08:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245910+0000", 2021-01-15T19:08:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3360000000000001e-05 2021-01-15T19:08:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245924+0000", 2021-01-15T19:08:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8702000000000001e-05 2021-01-15T19:08:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245952+0000", 2021-01-15T19:08:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9072e-05 2021-01-15T19:08:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245971+0000", 2021-01-15T19:08:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6116000000000002e-05 2021-01-15T19:08:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246037+0000", 2021-01-15T19:08:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029719949999999998 2021-01-15T19:08:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249009+0000", 2021-01-15T19:08:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031852109999999999 2021-01-15T19:08:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=34, last_complete=26'12, committed=1, applied=1))", 2021-01-15T19:08:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.245994+0000", 2021-01-15T19:08:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5743291290000001, 2021-01-15T19:08:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047559530000000003, 2021-01-15T19:08:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245994+0000", 2021-01-15T19:08:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245994+0000", 2021-01-15T19:08:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5293000000000001e-05 2021-01-15T19:08:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246049+0000", 2021-01-15T19:08:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6680000000000007e-06 2021-01-15T19:08:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246058+0000", 2021-01-15T19:08:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3308e-05 2021-01-15T19:08:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246071+0000", 2021-01-15T19:08:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.798e-06 2021-01-15T19:08:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246078+0000", 2021-01-15T19:08:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046489740000000002 2021-01-15T19:08:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250727+0000", 2021-01-15T19:08:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2912e-05 2021-01-15T19:08:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250750+0000", 2021-01-15T19:08:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047559530000000003 2021-01-15T19:08:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpRead(3.es1 26/21 ECSubRead(tid=26, to_read={3:72e8704c:::smithi00140323-12:head=0,94208,0}, subchunks={3:72e8704c:::smithi00140323-12:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.246615+0000", 2021-01-15T19:08:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.573707543, 2021-01-15T19:08:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026341200000000002, 2021-01-15T19:08:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246615+0000", 2021-01-15T19:08:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246615+0000", 2021-01-15T19:08:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3114000000000004e-05 2021-01-15T19:08:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246699+0000", 2021-01-15T19:08:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4994000000000001e-05 2021-01-15T19:08:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246714+0000", 2021-01-15T19:08:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8124999999999999e-05 2021-01-15T19:08:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246742+0000", 2021-01-15T19:08:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1218e-05 2021-01-15T19:08:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246753+0000", 2021-01-15T19:08:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023001760000000001 2021-01-15T19:08:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249053+0000", 2021-01-15T19:08:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000196493 2021-01-15T19:08:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249250+0000", 2021-01-15T19:08:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026341200000000002 2021-01-15T19:08:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpRead(3.es1 26/21 ECSubRead(tid=27, to_read={3:72e8704c:::smithi00140323-12:head=0,94208,0}, subchunks={3:72e8704c:::smithi00140323-12:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.246833+0000", 2021-01-15T19:08:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.573489887, 2021-01-15T19:08:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003191887, 2021-01-15T19:08:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246833+0000", 2021-01-15T19:08:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246833+0000", 2021-01-15T19:08:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8665999999999995e-05 2021-01-15T19:08:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246912+0000", 2021-01-15T19:08:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1653e-05 2021-01-15T19:08:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246923+0000", 2021-01-15T19:08:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3291000000000001e-05 2021-01-15T19:08:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246947+0000", 2021-01-15T19:08:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:08:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246955+0000", 2021-01-15T19:08:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025254510000000002 2021-01-15T19:08:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249480+0000", 2021-01-15T19:08:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00054479099999999998 2021-01-15T19:08:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250025+0000", 2021-01-15T19:08:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003191887 2021-01-15T19:08:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=36, last_complete=26'13, committed=1, applied=1))", 2021-01-15T19:08:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.249149+0000", 2021-01-15T19:08:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5711735600000001, 2021-01-15T19:08:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001668573, 2021-01-15T19:08:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249149+0000", 2021-01-15T19:08:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249149+0000", 2021-01-15T19:08:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3131e-05 2021-01-15T19:08:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249243+0000", 2021-01-15T19:08:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004e-05 2021-01-15T19:08:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249253+0000", 2021-01-15T19:08:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3522e-05 2021-01-15T19:08:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249266+0000", 2021-01-15T19:08:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1212999999999999e-05 2021-01-15T19:08:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249277+0000", 2021-01-15T19:08:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015226459999999999 2021-01-15T19:08:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250800+0000", 2021-01-15T19:08:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8020999999999999e-05 2021-01-15T19:08:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250818+0000", 2021-01-15T19:08:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001668573 2021-01-15T19:08:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 26/21 ECSubWriteReply(tid=35, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.249746+0000", 2021-01-15T19:08:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5705767580000001, 2021-01-15T19:08:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076153199999999999, 2021-01-15T19:08:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249746+0000", 2021-01-15T19:08:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249746+0000", 2021-01-15T19:08:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5275000000000002e-05 2021-01-15T19:08:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249802+0000", 2021-01-15T19:08:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.645e-06 2021-01-15T19:08:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249811+0000", 2021-01-15T19:08:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9984e-05 2021-01-15T19:08:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249831+0000", 2021-01-15T19:08:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1510000000000001e-06 2021-01-15T19:08:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249837+0000", 2021-01-15T19:08:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00065704299999999995 2021-01-15T19:08:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250494+0000", 2021-01-15T19:08:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3434e-05 2021-01-15T19:08:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250508+0000", 2021-01-15T19:08:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076153199999999999 2021-01-15T19:08:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=34, last_complete=26'12, committed=1, applied=1))", 2021-01-15T19:08:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.250662+0000", 2021-01-15T19:08:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5696614369999999, 2021-01-15T19:08:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036457899999999999, 2021-01-15T19:08:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250662+0000", 2021-01-15T19:08:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250662+0000", 2021-01-15T19:08:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003366815 2021-01-15T19:08:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254028+0000", 2021-01-15T19:08:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1878e-05 2021-01-15T19:08:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254040+0000", 2021-01-15T19:08:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4190000000000008e-06 2021-01-15T19:08:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254050+0000", 2021-01-15T19:08:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1590000000000003e-06 2021-01-15T19:08:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254054+0000", 2021-01-15T19:08:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3614e-05 2021-01-15T19:08:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254097+0000", 2021-01-15T19:08:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000209905 2021-01-15T19:08:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254307+0000", 2021-01-15T19:08:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036457899999999999 2021-01-15T19:08:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.2s2 26/21 ECSubWrite(tid=60, reqid=client.4418.0:166, at_version=26'8, trim_to=0'0, roll_forward_to=25'6))", 2021-01-15T19:08:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5674804200000001, 2021-01-15T19:08:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053879080000000003, 2021-01-15T19:08:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000255274 2021-01-15T19:08:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253098+0000", 2021-01-15T19:08:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6628999999999997e-05 2021-01-15T19:08:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253145+0000", 2021-01-15T19:08:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7581e-05 2021-01-15T19:08:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253162+0000", 2021-01-15T19:08:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3610000000000003e-06 2021-01-15T19:08:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253169+0000", 2021-01-15T19:08:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2859000000000003e-05 2021-01-15T19:08:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253222+0000", 2021-01-15T19:08:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2021000000000001e-05 2021-01-15T19:08:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253234+0000", 2021-01-15T19:08:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049458130000000003 2021-01-15T19:08:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258180+0000", 2021-01-15T19:08:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0370000000000001e-05 2021-01-15T19:08:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258231+0000", 2021-01-15T19:08:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053879080000000003 2021-01-15T19:08:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=37, last_complete=26'14, committed=1, applied=1))", 2021-01-15T19:08:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.255288+0000", 2021-01-15T19:08:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.565035322, 2021-01-15T19:08:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041825289999999999, 2021-01-15T19:08:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255288+0000", 2021-01-15T19:08:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255288+0000", 2021-01-15T19:08:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3288999999999998e-05 2021-01-15T19:08:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255341+0000", 2021-01-15T19:08:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1599999999999998e-06 2021-01-15T19:08:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255349+0000", 2021-01-15T19:08:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2616000000000001e-05 2021-01-15T19:08:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255362+0000", 2021-01-15T19:08:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4280000000000004e-06 2021-01-15T19:08:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255367+0000", 2021-01-15T19:08:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3349000000000001e-05 2021-01-15T19:08:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255411+0000", 2021-01-15T19:08:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040596870000000002 2021-01-15T19:08:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259470+0000", 2021-01-15T19:08:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041825289999999999 2021-01-15T19:08:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.2s2 26/21 ECSubWrite(tid=61, reqid=client.4418.0:167, at_version=26'9, trim_to=0'0, roll_forward_to=25'6))", 2021-01-15T19:08:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.255821+0000", 2021-01-15T19:08:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5645017320000001, 2021-01-15T19:08:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027540770000000002, 2021-01-15T19:08:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255821+0000", 2021-01-15T19:08:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255821+0000", 2021-01-15T19:08:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021174200000000001 2021-01-15T19:08:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.256033+0000", 2021-01-15T19:08:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1282999999999999e-05 2021-01-15T19:08:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.256054+0000", 2021-01-15T19:08:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6324000000000002e-05 2021-01-15T19:08:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.256071+0000", 2021-01-15T19:08:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6810000000000001e-06 2021-01-15T19:08:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.256077+0000", 2021-01-15T19:08:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031045299999999998 2021-01-15T19:08:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.256388+0000", 2021-01-15T19:08:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0084e-05 2021-01-15T19:08:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.256398+0000", 2021-01-15T19:08:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021502040000000002 2021-01-15T19:08:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258548+0000", 2021-01-15T19:08:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7305999999999999e-05 2021-01-15T19:08:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258575+0000", 2021-01-15T19:08:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027540770000000002 2021-01-15T19:08:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.2s2 26/21 ECSubWrite(tid=63, reqid=client.4418.0:168, at_version=26'10, trim_to=0'0, roll_forward_to=25'6))", 2021-01-15T19:08:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.257545+0000", 2021-01-15T19:08:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.562777783, 2021-01-15T19:08:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011306070000000001, 2021-01-15T19:08:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257545+0000", 2021-01-15T19:08:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257545+0000", 2021-01-15T19:08:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3019000000000002e-05 2021-01-15T19:08:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257598+0000", 2021-01-15T19:08:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9565e-05 2021-01-15T19:08:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257618+0000", 2021-01-15T19:08:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5248e-05 2021-01-15T19:08:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257633+0000", 2021-01-15T19:08:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.472e-06 2021-01-15T19:08:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257640+0000", 2021-01-15T19:08:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106016 2021-01-15T19:08:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257746+0000", 2021-01-15T19:08:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2823000000000001e-05 2021-01-15T19:08:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.257758+0000", 2021-01-15T19:08:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00087517300000000003 2021-01-15T19:08:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258634+0000", 2021-01-15T19:08:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2290999999999999e-05 2021-01-15T19:08:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258676+0000", 2021-01-15T19:08:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011306070000000001 2021-01-15T19:08:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpRead(3.cs0 26/21 ECSubRead(tid=40, to_read={3:3ef45b66:::smithi00140323-11:head=0,4096,0}, subchunks={3:3ef45b66:::smithi00140323-11:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.258389+0000", 2021-01-15T19:08:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.561933639, 2021-01-15T19:08:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00046353200000000001, 2021-01-15T19:08:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258389+0000", 2021-01-15T19:08:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258389+0000", 2021-01-15T19:08:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258389+0000", 2021-01-15T19:08:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258389+0000", 2021-01-15T19:08:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100709 2021-01-15T19:08:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258490+0000", 2021-01-15T19:08:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2139999999999999e-06 2021-01-15T19:08:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258497+0000", 2021-01-15T19:08:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023546 2021-01-15T19:08:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258733+0000", 2021-01-15T19:08:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120149 2021-01-15T19:08:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258853+0000", 2021-01-15T19:08:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00046353200000000001 2021-01-15T19:08:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpReadReply(3.cs0 26/21 ECSubReadReply(tid=40, attrs_read=0))", 2021-01-15T19:08:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.259882+0000", 2021-01-15T19:08:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.560441432, 2021-01-15T19:08:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028011699999999997, 2021-01-15T19:08:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259882+0000", 2021-01-15T19:08:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259882+0000", 2021-01-15T19:08:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7532999999999999e-05 2021-01-15T19:08:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259909+0000", 2021-01-15T19:08:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259924+0000", 2021-01-15T19:08:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.066e-05 2021-01-15T19:08:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259934+0000", 2021-01-15T19:08:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6849999999999999e-06 2021-01-15T19:08:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259938+0000", 2021-01-15T19:08:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6179e-05 2021-01-15T19:08:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259974+0000", 2021-01-15T19:08:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018741 2021-01-15T19:08:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260162+0000", 2021-01-15T19:08:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028011699999999997 2021-01-15T19:08:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.2s2 26/21 ECSubWrite(tid=66, reqid=client.4418.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=26'10))", 2021-01-15T19:08:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.260359+0000", 2021-01-15T19:08:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5599638149999999, 2021-01-15T19:08:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058625500000000004, 2021-01-15T19:08:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260359+0000", 2021-01-15T19:08:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260359+0000", 2021-01-15T19:08:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4997999999999999e-05 2021-01-15T19:08:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260394+0000", 2021-01-15T19:08:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0672999999999999e-05 2021-01-15T19:08:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260405+0000", 2021-01-15T19:08:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8437999999999999e-05 2021-01-15T19:08:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260423+0000", 2021-01-15T19:08:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3710000000000006e-06 2021-01-15T19:08:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260433+0000", 2021-01-15T19:08:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3114000000000001e-05 2021-01-15T19:08:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260476+0000", 2021-01-15T19:08:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0784e-05 2021-01-15T19:08:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260487+0000", 2021-01-15T19:08:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040664299999999999 2021-01-15T19:08:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260893+0000", 2021-01-15T19:08:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2234000000000002e-05 2021-01-15T19:08:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260945+0000", 2021-01-15T19:08:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058625500000000004 2021-01-15T19:08:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:16.084 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:16.085 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:16.084+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:16.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:16.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:16.186 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:16.184+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:16.249 INFO:tasks.rados.rados.0.smithi001.stdout:71: done (4 left) 2021-01-15T19:08:16.249 INFO:tasks.rados.rados.0.smithi001.stdout:72: read oid 3 snap 1 2021-01-15T19:08:16.249 INFO:tasks.rados.rados.0.smithi001.stdout:72: expect (ObjNum 2 snap 0 seq_num 2) 2021-01-15T19:08:16.249 INFO:tasks.rados.rados.0.smithi001.stdout:73: read oid 34 snap -1 2021-01-15T19:08:16.250 INFO:tasks.rados.rados.0.smithi001.stdout:73: expect (ObjNum 33 snap 0 seq_num 33) 2021-01-15T19:08:16.250 INFO:tasks.rados.rados.0.smithi001.stdout:74: delete oid 33 current snap is 2 2021-01-15T19:08:16.250 INFO:tasks.rados.rados.0.smithi001.stdout:68: finishing rollback tid 0 to smithi00140323-43 2021-01-15T19:08:16.250 INFO:tasks.rados.rados.0.smithi001.stdout:68: finishing rollback tid 1 to smithi00140323-43 2021-01-15T19:08:16.251 INFO:tasks.rados.rados.0.smithi001.stdout:66: finishing write tid 1 to smithi00140323-39 2021-01-15T19:08:16.251 INFO:tasks.rados.rados.0.smithi001.stdout:68: finishing rollback tid 2 to smithi00140323-43 2021-01-15T19:08:16.251 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 14 (ObjNum 42 snap 0 seq_num 42) dirty exists 2021-01-15T19:08:16.251 INFO:tasks.rados.rados.0.smithi001.stdout:70: finishing write tid 1 to smithi00140323-14 2021-01-15T19:08:16.251 INFO:tasks.rados.rados.0.smithi001.stdout:70: finishing write tid 2 to smithi00140323-14 2021-01-15T19:08:16.252 INFO:tasks.rados.rados.0.smithi001.stdout:70: finishing write tid 3 to smithi00140323-14 2021-01-15T19:08:16.252 INFO:tasks.rados.rados.0.smithi001.stdout:70: finishing write tid 4 to smithi00140323-14 2021-01-15T19:08:16.252 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 12 (ObjNum 55 snap 2 seq_num 55) dirty exists 2021-01-15T19:08:16.252 INFO:tasks.rados.rados.0.smithi001.stdout:70: left oid 14 (ObjNum 55 snap 2 seq_num 55) 2021-01-15T19:08:16.252 INFO:tasks.rados.rados.0.smithi001.stdout:67: expect (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:08:16.253 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:08:16.278 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:16.279 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:16.286 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:16.285+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:16.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:16.295 INFO:tasks.rados.rados.0.smithi001.stdout:66: finishing write tid 2 to smithi00140323-39 2021-01-15T19:08:16.296 INFO:tasks.rados.rados.0.smithi001.stdout:66: finishing write tid 3 to smithi00140323-39 2021-01-15T19:08:16.296 INFO:tasks.rados.rados.0.smithi001.stdout:66: finishing write tid 4 to smithi00140323-39 2021-01-15T19:08:16.296 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 10 (ObjNum 54 snap 2 seq_num 54) dirty exists 2021-01-15T19:08:16.296 INFO:tasks.rados.rados.0.smithi001.stdout:66: left oid 39 (ObjNum 54 snap 2 seq_num 54) 2021-01-15T19:08:16.297 INFO:tasks.rados.rados.0.smithi001.stdout:72: expect (ObjNum 2 snap 0 seq_num 2) 2021-01-15T19:08:16.297 INFO:tasks.rados.rados.0.smithi001.stdout:73: expect (ObjNum 33 snap 0 seq_num 33) 2021-01-15T19:08:16.298 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:16.312 INFO:tasks.rados.rados.0.smithi001.stdout:66: done (6 left) 2021-01-15T19:08:16.312 INFO:tasks.rados.rados.0.smithi001.stdout:67: done (5 left) 2021-01-15T19:08:16.313 INFO:tasks.rados.rados.0.smithi001.stdout:68: done (4 left) 2021-01-15T19:08:16.313 INFO:tasks.rados.rados.0.smithi001.stdout:70: done (3 left) 2021-01-15T19:08:16.313 INFO:tasks.rados.rados.0.smithi001.stdout:72: done (2 left) 2021-01-15T19:08:16.314 INFO:tasks.rados.rados.0.smithi001.stdout:73: done (1 left) 2021-01-15T19:08:16.314 INFO:tasks.rados.rados.0.smithi001.stdout:74: done (0 left) 2021-01-15T19:08:16.314 INFO:tasks.rados.rados.0.smithi001.stdout:75: read oid 9 snap 1 2021-01-15T19:08:16.314 INFO:tasks.rados.rados.0.smithi001.stdout:75: expect (ObjNum 8 snap 0 seq_num 8) 2021-01-15T19:08:16.315 INFO:tasks.rados.rados.0.smithi001.stdout:76: read oid 19 snap -1 2021-01-15T19:08:16.315 INFO:tasks.rados.rados.0.smithi001.stdout:76: expect (ObjNum 18 snap 0 seq_num 18) 2021-01-15T19:08:16.315 INFO:tasks.rados.rados.0.smithi001.stdout:77: snap_create 2021-01-15T19:08:16.315 INFO:tasks.rados.rados.0.smithi001.stdout:75: expect (ObjNum 8 snap 0 seq_num 8) 2021-01-15T19:08:16.319 INFO:tasks.rados.rados.0.smithi001.stdout:76: expect (ObjNum 18 snap 0 seq_num 18) 2021-01-15T19:08:16.319 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:16.339 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:16.359 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:16.380 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:16.387 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:16.386+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:16.401 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:16.422 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:16.431 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:16.431 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:16.441 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:16.443 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:16.496 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:16.495+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:16.575 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:16.575 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:16.586 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:16.628 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:16.625+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:16.693 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:16.691+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:16.759 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:16.760 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:16.761 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:16.759+0000 7f0cb39e9700 -1 osd.4 pg_epoch: 28 pg[3.6s0( v 26'11 (0'0,26'11] local-lis/les=21/22 n=5 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) [4,7,1]p4(0) r=0 lpr=21 crt=26'11 lcod 25'9 mlcod 25'9 active+clean+snaptrim trimq=[2~1]] removing snap head 2021-01-15T19:08:16.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:16.821 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:16.815+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:16.895 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:16.894+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:16.904 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:16.904 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:16.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:16.999 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:16.997+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:17.086 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:17.086 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:17.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:17.098 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:17.097+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:17.200 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:17.198+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:17.253 INFO:tasks.rados.rados.0.smithi001.stdout:75: done (2 left) 2021-01-15T19:08:17.253 INFO:tasks.rados.rados.0.smithi001.stdout:76: done (1 left) 2021-01-15T19:08:17.254 INFO:tasks.rados.rados.0.smithi001.stdout:77: done (0 left) 2021-01-15T19:08:17.254 INFO:tasks.rados.rados.0.smithi001.stdout:78: setattr oid 20 current snap is 3 2021-01-15T19:08:17.254 INFO:tasks.rados.rados.0.smithi001.stdout:79: read oid 22 snap 2 2021-01-15T19:08:17.254 INFO:tasks.rados.rados.0.smithi001.stdout:79: expect (ObjNum 21 snap 0 seq_num 21) 2021-01-15T19:08:17.255 INFO:tasks.rados.rados.0.smithi001.stdout:80: copy_from oid 39 from oid 41 current snap is 3 2021-01-15T19:08:17.255 INFO:tasks.rados.rados.0.smithi001.stdout:81: snap_create 2021-01-15T19:08:17.258 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 20 v 8 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:08:17.261 INFO:tasks.rados.rados.0.smithi001.stdout:79: expect (ObjNum 21 snap 0 seq_num 21) 2021-01-15T19:08:17.272 INFO:tasks.rados.rados.0.smithi001.stdout:80: finishing copy_from to smithi00140323-39 2021-01-15T19:08:17.272 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2021-01-15T19:08:17.273 INFO:tasks.rados.rados.0.smithi001.stdout:80: finishing copy_from racing read to smithi00140323-39 2021-01-15T19:08:17.276 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:17.276 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:17.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:17.331 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:17.329+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:17.402 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:17.400+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:17.418 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:17.418 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:17.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:17.502 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:17.501+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:17.599 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:17.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:17.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:17.603 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:17.603 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:17.603 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:17.600+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:17.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:17.703 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:17.701+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:17.750 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:17.750 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:17.759 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:17.804 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:17.802+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:17.884 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:17.885 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:17.893 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:17.905 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:17.903+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:17.911 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:17.912 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:17.923 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2021-01-15T19:08:17.509587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26034176,"num_objects":60,"num_object_clones":8,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":2,"num_read":132,"num_read_kb":8918,"num_write":146,"num_write_kb":25457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78080,"kb_used_data":37696,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754896640,"statfs":{"total":773094113280,"available":773014159360,"internally_reserved":0,"allocated":38600704,"data_stored":41053761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26034176,"num_objects":58,"num_object_clones":8,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":2,"num_read":132,"num_read_kb":8918,"num_write":144,"num_write_kb":25448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000671"},"pg_stats":[{"pgid":"3.f","version":"28'10","reported_seq":"32","reported_epoch":"28","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:16.313667+0000","last_change":"2021-01-15T19:08:16.250986+0000","last_active":"2021-01-15T19:08:16.313667+0000","last_peered":"2021-01-15T19:08:16.313667+0000","last_clean":"2021-01-15T19:08:16.313667+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:16.313667+0000","last_undegraded":"2021-01-15T19:08:16.313667+0000","last_fullsized":"2021-01-15T19:08:16.313667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":1687552,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":892,"num_write":7,"num_write_kb":1672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'4","reported_seq":"22","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.250767+0000","last_change":"2021-01-15T19:08:10.234384+0000","last_active":"2021-01-15T19:08:14.250767+0000","last_peered":"2021-01-15T19:08:14.250767+0000","last_clean":"2021-01-15T19:08:14.250767+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:14.250767+0000","last_undegraded":"2021-01-15T19:08:14.250767+0000","last_fullsized":"2021-01-15T19:08:14.250767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":555,"num_write":4,"num_write_kb":848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"25'6","reported_seq":"27","reported_epoch":"28","state":"active+clean","last_fresh":"2021-01-15T19:08:16.262094+0000","last_change":"2021-01-15T19:08:16.263603+0000","last_active":"2021-01-15T19:08:16.262094+0000","last_peered":"2021-01-15T19:08:16.262094+0000","last_clean":"2021-01-15T19:08:16.262094+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:16.262094+0000","last_undegraded":"2021-01-15T19:08:16.262094+0000","last_fullsized":"2021-01-15T19:08:16.262094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":417792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"28'18","reported_seq":"41","reported_epoch":"28","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:16.263369+0000","last_change":"2021-01-15T19:08:16.256637+0000","last_active":"2021-01-15T19:08:16.263369+0000","last_peered":"2021-01-15T19:08:16.263369+0000","last_clean":"2021-01-15T19:08:16.263369+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:16.263369+0000","last_undegraded":"2021-01-15T19:08:16.263369+0000","last_fullsized":"2021-01-15T19:08:16.263369+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":2965504,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":16,"num_read_kb":1639,"num_write":14,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"25'4","reported_seq":"25","reported_epoch":"28","state":"active+clean","last_fresh":"2021-01-15T19:08:16.247577+0000","last_change":"2021-01-15T19:08:16.247836+0000","last_active":"2021-01-15T19:08:16.247577+0000","last_peered":"2021-01-15T19:08:16.247577+0000","last_clean":"2021-01-15T19:08:16.247577+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:16.247577+0000","last_undegraded":"2021-01-15T19:08:16.247577+0000","last_fullsized":"2021-01-15T19:08:16.247577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1220608,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":1192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"25'6","reported_seq":"28","reported_epoch":"28","state":"active+clean","last_fresh":"2021-01-15T19:08:16.260643+0000","last_change":"2021-01-15T19:08:16.260955+0000","last_active":"2021-01-15T19:08:16.260643+0000","last_peered":"2021-01-15T19:08:16.260643+0000","last_clean":"2021-01-15T19:08:16.260643+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:16.260643+0000","last_undegraded":"2021-01-15T19:08:16.260643+0000","last_fullsized":"2021-01-15T19:08:16.260643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1843200,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"28'17","reported_seq":"41","reported_epoch":"28","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:16.260570+0000","last_change":"2021-01-15T19:08:16.260570+0000","last_active":"2021-01-15T19:08:16.260570+0000","last_peered":"2021-01-15T19:08:16.260570+0000","last_clean":"2021-01-15T19:08:16.260570+0000","last_became_active":"2021-01-15T19:08:10.230943+0000","last_became_peered":"2021-01-15T19:08:10.230943+0000","last_unstale":"2021-01-15T19:08:16.260570+0000","last_undegraded":"2021-01-15T19:08:16.260570+0000","last_fullsized":"2021-01-15T19:08:16.260570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":3129344,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":1831,"num_write":14,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"27'14","reported_seq":"33","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.256707+0000","last_change":"2021-01-15T19:08:10.231854+0000","last_active":"2021-01-15T19:08:15.256707+0000","last_peered":"2021-01-15T19:08:15.256707+0000","last_clean":"2021-01-15T19:08:15.256707+0000","last_became_active":"2021-01-15T19:08:10.231245+0000","last_became_peered":"2021-01-15T19:08:10.231245+0000","last_unstale":"2021-01-15T19:08:15.256707+0000","last_undegraded":"2021-01-15T19:08:15.256707+0000","last_fullsized":"2021-01-15T19:08:15.256707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2547712,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":12,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"17","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.282014+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:08:14.282014+0000","last_peered":"2021-01-15T19:08:14.282014+0000","last_clean":"2021-01-15T19:08:14.282014+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:08:14.282014+0000","last_undegraded":"2021-01-15T19:08:14.282014+0000","last_fullsized":"2021-01-15T19:08:14.282014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"22","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.236927+0000","last_change":"2021-01-15T19:08:10.231323+0000","last_active":"2021-01-15T19:08:14.236927+0000","last_peered":"2021-01-15T19:08:14.236927+0000","last_clean":"2021-01-15T19:08:14.236927+0000","last_became_active":"2021-01-15T19:08:10.230610+0000","last_became_peered":"2021-01-15T19:08:10.230610+0000","last_unstale":"2021-01-15T19:08:14.236927+0000","last_undegraded":"2021-01-15T19:08:14.236927+0000","last_fullsized":"2021-01-15T19:08:14.236927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1548288,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"17","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.334857+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:08:15.334857+0000","last_peered":"2021-01-15T19:08:15.334857+0000","last_clean":"2021-01-15T19:08:15.334857+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:08:15.334857+0000","last_undegraded":"2021-01-15T19:08:15.334857+0000","last_fullsized":"2021-01-15T19:08:15.334857+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"18","reported_epoch":"28","state":"active+clean","last_fresh":"2021-01-15T19:08:16.250885+0000","last_change":"2021-01-15T19:08:16.251239+0000","last_active":"2021-01-15T19:08:16.250885+0000","last_peered":"2021-01-15T19:08:16.250885+0000","last_clean":"2021-01-15T19:08:16.250885+0000","last_became_active":"2021-01-15T19:08:10.237812+0000","last_became_peered":"2021-01-15T19:08:10.237812+0000","last_unstale":"2021-01-15T19:08:16.250885+0000","last_undegraded":"2021-01-15T19:08:16.250885+0000","last_fullsized":"2021-01-15T19:08:16.250885+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"19","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.242250+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:08:15.242250+0000","last_peered":"2021-01-15T19:08:15.242250+0000","last_clean":"2021-01-15T19:08:15.242250+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:08:15.242250+0000","last_undegraded":"2021-01-15T19:08:15.242250+0000","last_fullsized":"2021-01-15T19:08:15.242250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'10","reported_seq":"32","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.285232+0000","last_change":"2021-01-15T19:08:10.231670+0000","last_active":"2021-01-15T19:08:14.285232+0000","last_peered":"2021-01-15T19:08:14.285232+0000","last_clean":"2021-01-15T19:08:14.285232+0000","last_became_active":"2021-01-15T19:08:10.231211+0000","last_became_peered":"2021-01-15T19:08:10.231211+0000","last_unstale":"2021-01-15T19:08:14.285232+0000","last_undegraded":"2021-01-15T19:08:14.285232+0000","last_fullsized":"2021-01-15T19:08:14.285232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1523712,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":750,"num_write":10,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"17","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.334861+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:08:15.334861+0000","last_peered":"2021-01-15T19:08:15.334861+0000","last_clean":"2021-01-15T19:08:15.334861+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:08:15.334861+0000","last_undegraded":"2021-01-15T19:08:15.334861+0000","last_fullsized":"2021-01-15T19:08:15.334861+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"23","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.241931+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:14.241931+0000","last_peered":"2021-01-15T19:08:14.241931+0000","last_clean":"2021-01-15T19:08:14.241931+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:14.241931+0000","last_undegraded":"2021-01-15T19:08:14.241931+0000","last_fullsized":"2021-01-15T19:08:14.241931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"26'10","reported_seq":"30","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.257129+0000","last_change":"2021-01-15T19:08:10.234902+0000","last_active":"2021-01-15T19:08:15.257129+0000","last_peered":"2021-01-15T19:08:15.257129+0000","last_clean":"2021-01-15T19:08:15.257129+0000","last_became_active":"2021-01-15T19:08:10.234192+0000","last_became_peered":"2021-01-15T19:08:10.234192+0000","last_unstale":"2021-01-15T19:08:15.257129+0000","last_undegraded":"2021-01-15T19:08:15.257129+0000","last_fullsized":"2021-01-15T19:08:15.257129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1974272,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":3221,"num_write":9,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"17","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.334811+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:08:15.334811+0000","last_peered":"2021-01-15T19:08:15.334811+0000","last_clean":"2021-01-15T19:08:15.334811+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:08:15.334811+0000","last_undegraded":"2021-01-15T19:08:15.334811+0000","last_fullsized":"2021-01-15T19:08:15.334811+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'12","reported_seq":"28","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.270691+0000","last_change":"2021-01-15T19:08:10.230992+0000","last_active":"2021-01-15T19:08:15.270691+0000","last_peered":"2021-01-15T19:08:15.270691+0000","last_clean":"2021-01-15T19:08:15.270691+0000","last_became_active":"2021-01-15T19:08:10.230297+0000","last_became_peered":"2021-01-15T19:08:10.230297+0000","last_unstale":"2021-01-15T19:08:15.270691+0000","last_undegraded":"2021-01-15T19:08:15.270691+0000","last_fullsized":"2021-01-15T19:08:15.270691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1966080,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":29,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"17","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.334991+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:08:15.334991+0000","last_peered":"2021-01-15T19:08:15.334991+0000","last_clean":"2021-01-15T19:08:15.334991+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:08:15.334991+0000","last_undegraded":"2021-01-15T19:08:15.334991+0000","last_fullsized":"2021-01-15T19:08:15.334991+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'6","reported_seq":"23","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.257359+0000","last_change":"2021-01-15T19:08:10.228930+0000","last_active":"2021-01-15T19:08:14.257359+0000","last_peered":"2021-01-15T19:08:14.257359+0000","last_clean":"2021-01-15T19:08:14.257359+0000","last_became_active":"2021-01-15T19:08:10.228436+0000","last_became_peered":"2021-01-15T19:08:10.228436+0000","last_unstale":"2021-01-15T19:08:14.257359+0000","last_undegraded":"2021-01-15T19:08:14.257359+0000","last_fullsized":"2021-01-15T19:08:14.257359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1163264,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"27'10","reported_seq":"24","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.262760+0000","last_change":"2021-01-15T19:08:10.704412+0000","last_active":"2021-01-15T19:08:15.262760+0000","last_peered":"2021-01-15T19:08:15.262760+0000","last_clean":"2021-01-15T19:08:15.262760+0000","last_became_active":"2021-01-15T19:08:10.703898+0000","last_became_peered":"2021-01-15T19:08:10.703898+0000","last_unstale":"2021-01-15T19:08:15.262760+0000","last_undegraded":"2021-01-15T19:08:15.262760+0000","last_fullsized":"2021-01-15T19:08:15.262760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1548288,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":7,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.246842+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:08:14.246842+0000","last_peered":"2021-01-15T19:08:14.246842+0000","last_clean":"2021-01-15T19:08:14.246842+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:08:14.246842+0000","last_undegraded":"2021-01-15T19:08:14.246842+0000","last_fullsized":"2021-01-15T19:08:14.246842+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"17","reported_epoch":"27","state":"active+clean","last_fresh":"2021-01-15T19:08:15.334941+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:08:15.334941+0000","last_peered":"2021-01-15T19:08:15.334941+0000","last_clean":"2021-01-15T19:08:15.334941+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:08:15.334941+0000","last_undegraded":"2021-01-15T19:08:15.334941+0000","last_fullsized":"2021-01-15T19:08:15.334941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'11","reported_seq":"28","reported_epoch":"26","state":"active+clean","last_fresh":"2021-01-15T19:08:14.279871+0000","last_change":"2021-01-15T19:08:10.704433+0000","last_active":"2021-01-15T19:08:14.279871+0000","last_peered":"2021-01-15T19:08:14.279871+0000","last_clean":"2021-01-15T19:08:14.279871+0000","last_became_active":"2021-01-15T19:08:10.703953+0000","last_became_peered":"2021-01-15T19:08:10.703953+0000","last_unstale":"2021-01-15T19:08:14.279871+0000","last_undegraded":"2021-01-15T19:08:14.279871+0000","last_fullsized":"2021-01-15T19:08:14.279871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1630208,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":8,"num_read_kb":4,"num_write":10,"num_write_kb":1592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":26034176,"num_objects":58,"num_object_clones":8,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":2,"num_read":132,"num_read_kb":8918,"num_write":144,"num_write_kb":25448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39075840,"data_stored":44097536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11076,"kb_used_data":6012,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94360764,"statfs":{"total":96636764160,"available":96625422336,"internally_reserved":0,"allocated":6156288,"data_stored":6425500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":3012,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":3084288,"data_stored":3449417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8320,"kb_used_data":3320,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94363520,"statfs":{"total":96636764160,"available":96628244480,"internally_reserved":0,"allocated":3399680,"data_stored":3551817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6559999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9552,"kb_used_data":4552,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94362288,"statfs":{"total":96636764160,"available":96626982912,"internally_reserved":0,"allocated":4661248,"data_stored":4692892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8240000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11568,"kb_used_data":6440,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94360272,"statfs":{"total":96636764160,"available":96624918528,"internally_reserved":0,"allocated":6594560,"data_stored":7172681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9364,"kb_used_data":4364,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94362476,"statfs":{"total":96636764160,"available":96627175424,"internally_reserved":0,"allocated":4468736,"data_stored":4795292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13180,"kb_used_data":7988,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94358660,"statfs":{"total":96636764160,"available":96623267840,"internally_reserved":0,"allocated":8179712,"data_stored":9077321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0640000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.901}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.52}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6944,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94364896,"statfs":{"total":96636764160,"available":96629653504,"internally_reserved":0,"allocated":2056192,"data_stored":1888841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]}]}} 2021-01-15T19:08:18.007 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:18.005+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:18.059 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:18.060 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:18.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:18.108 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:18.106+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:18.209 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:18.207+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:18.237 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:18.238 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:18.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:18.254 INFO:tasks.rados.rados.0.smithi001.stdout:78: done (3 left) 2021-01-15T19:08:18.254 INFO:tasks.rados.rados.0.smithi001.stdout:79: done (2 left) 2021-01-15T19:08:18.255 INFO:tasks.rados.rados.0.smithi001.stdout:80: done (1 left) 2021-01-15T19:08:18.255 INFO:tasks.rados.rados.0.smithi001.stdout:81: done (0 left) 2021-01-15T19:08:18.255 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 19 current snap is 4 2021-01-15T19:08:18.256 INFO:tasks.rados.rados.0.smithi001.stdout:82: seq_num 57 ranges {303104=557056,860160=401408} 2021-01-15T19:08:18.259 INFO:tasks.rados.rados.0.smithi001.stdout:82: writing smithi00140323-19 from 303104 to 860160 tid 1 2021-01-15T19:08:18.262 INFO:tasks.rados.rados.0.smithi001.stdout:82: writing smithi00140323-19 from 860160 to 1261568 tid 2 2021-01-15T19:08:18.263 INFO:tasks.rados.rados.0.smithi001.stdout:83: snap_create 2021-01-15T19:08:18.267 INFO:tasks.rados.rados.0.smithi001.stdout:82: finishing write tid 1 to smithi00140323-19 2021-01-15T19:08:18.268 INFO:tasks.rados.rados.0.smithi001.stdout:82: finishing write tid 2 to smithi00140323-19 2021-01-15T19:08:18.271 INFO:tasks.rados.rados.0.smithi001.stdout:82: finishing write tid 3 to smithi00140323-19 2021-01-15T19:08:18.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:08:18.273 INFO:tasks.rados.rados.0.smithi001.stdout:82: finishing write tid 4 to smithi00140323-19 2021-01-15T19:08:18.274 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 14 (ObjNum 57 snap 4 seq_num 57) dirty exists 2021-01-15T19:08:18.274 INFO:tasks.rados.rados.0.smithi001.stdout:82: left oid 19 (ObjNum 57 snap 4 seq_num 57) 2021-01-15T19:08:18.309 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:18.307+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:18.373 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:18.373 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:18.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:18.424 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:18.423+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:18.456 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:18.510 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:18.508+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:18.549 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:18.549 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:18.559 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:18.624 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:18.643 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:18.641+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:18.645 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:18.664 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:18.684 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:18.706 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:18.712 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:18.709+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:18.727 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:18.728 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:18.729 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:18.738 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:18.748 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.750 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.750 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.750 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.750 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.751 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:18.813 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:18.811+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:18.903 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:18.903 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:18.904 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:18.913 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:18.915 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:18.913+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:19.015 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:19.014+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:19.040 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:19.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=62, reqid=client.4418.0:170, at_version=26'4, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7815219290000002, 2021-01-15T19:08:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999, 2021-01-15T19:08:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:19.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:19.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530000000000001e-05 2021-01-15T19:08:19.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256020+0000", 2021-01-15T19:08:19.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4555000000000001e-05 2021-01-15T19:08:19.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256044+0000", 2021-01-15T19:08:19.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9589000000000002e-05 2021-01-15T19:08:19.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256064+0000", 2021-01-15T19:08:19.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1316e-05 2021-01-15T19:08:19.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256075+0000", 2021-01-15T19:08:19.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3517999999999996e-05 2021-01-15T19:08:19.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256149+0000", 2021-01-15T19:08:19.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2123999999999999e-05 2021-01-15T19:08:19.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256161+0000", 2021-01-15T19:08:19.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025493310000000002 2021-01-15T19:08:19.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258710+0000", 2021-01-15T19:08:19.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0933999999999997e-05 2021-01-15T19:08:19.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258761+0000", 2021-01-15T19:08:19.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999 2021-01-15T19:08:19.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.059 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.060 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.060 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=64, reqid=client.4418.0:171, at_version=26'5, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:19.060 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:19.061 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7799612949999997, 2021-01-15T19:08:19.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632711, 2021-01-15T19:08:19.061 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.062 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.062 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:19.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:19.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7207999999999999e-05 2021-01-15T19:08:19.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257573+0000", 2021-01-15T19:08:19.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2881e-05 2021-01-15T19:08:19.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257596+0000", 2021-01-15T19:08:19.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6541999999999998e-05 2021-01-15T19:08:19.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257613+0000", 2021-01-15T19:08:19.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4219999999999997e-06 2021-01-15T19:08:19.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257619+0000", 2021-01-15T19:08:19.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041151300000000002 2021-01-15T19:08:19.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258030+0000", 2021-01-15T19:08:19.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4518000000000001e-05 2021-01-15T19:08:19.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258045+0000", 2021-01-15T19:08:19.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001067995 2021-01-15T19:08:19.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259113+0000", 2021-01-15T19:08:19.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5631999999999999e-05 2021-01-15T19:08:19.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259159+0000", 2021-01-15T19:08:19.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632711 2021-01-15T19:08:19.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.079 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.080 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=65, reqid=client.4418.0:172, at_version=26'6, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:19.080 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:19.080 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7782833780000002, 2021-01-15T19:08:19.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093191900000000002, 2021-01-15T19:08:19.080 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.081 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:19.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:19.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9619e-05 2021-01-15T19:08:19.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259253+0000", 2021-01-15T19:08:19.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8291000000000002e-05 2021-01-15T19:08:19.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259272+0000", 2021-01-15T19:08:19.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6121e-05 2021-01-15T19:08:19.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259288+0000", 2021-01-15T19:08:19.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6279999999999999e-06 2021-01-15T19:08:19.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259294+0000", 2021-01-15T19:08:19.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4839000000000002e-05 2021-01-15T19:08:19.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259339+0000", 2021-01-15T19:08:19.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0299000000000001e-05 2021-01-15T19:08:19.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259350+0000", 2021-01-15T19:08:19.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073881299999999995 2021-01-15T19:08:19.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260088+0000", 2021-01-15T19:08:19.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7308999999999997e-05 2021-01-15T19:08:19.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260136+0000", 2021-01-15T19:08:19.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093191900000000002 2021-01-15T19:08:19.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.091 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.092 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=67, reqid=client.4418.0:172, at_version=0'0, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:19.092 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:19.092 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7768172440000001, 2021-01-15T19:08:19.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011872370000000001, 2021-01-15T19:08:19.093 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.093 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.093 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:19.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:19.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6808000000000003e-05 2021-01-15T19:08:19.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260717+0000", 2021-01-15T19:08:19.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3665e-05 2021-01-15T19:08:19.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260730+0000", 2021-01-15T19:08:19.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5318000000000001e-05 2021-01-15T19:08:19.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260746+0000", 2021-01-15T19:08:19.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4309999999999999e-06 2021-01-15T19:08:19.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260752+0000", 2021-01-15T19:08:19.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5484e-05 2021-01-15T19:08:19.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260798+0000", 2021-01-15T19:08:19.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4531e-05 2021-01-15T19:08:19.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260812+0000", 2021-01-15T19:08:19.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00098527100000000002 2021-01-15T19:08:19.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.261797+0000", 2021-01-15T19:08:19.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9728999999999997e-05 2021-01-15T19:08:19.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.261857+0000", 2021-01-15T19:08:19.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011872370000000001 2021-01-15T19:08:19.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.104 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.104 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:177 3.1s0 3:8f2f8304:::smithi00140323-26:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:19.105 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:19.105 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7559421869999996, 2021-01-15T19:08:19.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494, 2021-01-15T19:08:19.105 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.105 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:19.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:19.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:19.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:19.106 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2021-01-15T19:08:19.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.107 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:19.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:19.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6885000000000005e-05 2021-01-15T19:08:19.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281642+0000", 2021-01-15T19:08:19.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:08:19.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281650+0000", 2021-01-15T19:08:19.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2979999999999999e-05 2021-01-15T19:08:19.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281663+0000", 2021-01-15T19:08:19.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1930000000000001e-06 2021-01-15T19:08:19.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281669+0000", 2021-01-15T19:08:19.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880999999999999e-05 2021-01-15T19:08:19.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281724+0000", 2021-01-15T19:08:19.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946600000000001 2021-01-15T19:08:19.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:19.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281853+0000", 2021-01-15T19:08:19.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000582403 2021-01-15T19:08:19.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282435+0000", 2021-01-15T19:08:19.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013959510000000001 2021-01-15T19:08:19.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283831+0000", 2021-01-15T19:08:19.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010843630000000001 2021-01-15T19:08:19.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:19.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284916+0000", 2021-01-15T19:08:19.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5765000000000005e-05 2021-01-15T19:08:19.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284981+0000", 2021-01-15T19:08:19.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494 2021-01-15T19:08:19.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.119 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=27, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7536389589999999, 2021-01-15T19:08:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056741500000000004, 2021-01-15T19:08:19.120 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.121 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:19.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:19.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5211999999999998e-05 2021-01-15T19:08:19.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283873+0000", 2021-01-15T19:08:19.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2769999999999996e-06 2021-01-15T19:08:19.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283879+0000", 2021-01-15T19:08:19.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0971999999999999e-05 2021-01-15T19:08:19.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283890+0000", 2021-01-15T19:08:19.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1859999999999997e-06 2021-01-15T19:08:19.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283896+0000", 2021-01-15T19:08:19.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050626700000000002 2021-01-15T19:08:19.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284402+0000", 2021-01-15T19:08:19.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3501e-05 2021-01-15T19:08:19.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284416+0000", 2021-01-15T19:08:19.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056741500000000004 2021-01-15T19:08:19.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.129 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.130 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=27, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:19.130 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:19.130 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7528704069999996, 2021-01-15T19:08:19.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059160400000000002, 2021-01-15T19:08:19.131 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.131 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.131 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:19.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7327999999999998e-05 2021-01-15T19:08:19.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284664+0000", 2021-01-15T19:08:19.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8720000000000008e-06 2021-01-15T19:08:19.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284673+0000", 2021-01-15T19:08:19.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6592e-05 2021-01-15T19:08:19.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284690+0000", 2021-01-15T19:08:19.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.725e-06 2021-01-15T19:08:19.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284696+0000", 2021-01-15T19:08:19.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7620000000000006e-05 2021-01-15T19:08:19.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284764+0000", 2021-01-15T19:08:19.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044446699999999998 2021-01-15T19:08:19.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285208+0000", 2021-01-15T19:08:19.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059160400000000002 2021-01-15T19:08:19.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.139 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.140 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=71, reqid=client.4418.0:183, at_version=27'8, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:19.140 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:19.141 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7857904499999999, 2021-01-15T19:08:19.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003, 2021-01-15T19:08:19.141 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.141 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.141 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:19.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:19.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190392 2021-01-15T19:08:19.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251887+0000", 2021-01-15T19:08:19.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6526999999999999e-05 2021-01-15T19:08:19.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251914+0000", 2021-01-15T19:08:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8068999999999999e-05 2021-01-15T19:08:19.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251932+0000", 2021-01-15T19:08:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7269999999999996e-06 2021-01-15T19:08:19.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251940+0000", 2021-01-15T19:08:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012910800000000001 2021-01-15T19:08:19.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252070+0000", 2021-01-15T19:08:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2417e-05 2021-01-15T19:08:19.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252082+0000", 2021-01-15T19:08:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050434080000000001 2021-01-15T19:08:19.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257125+0000", 2021-01-15T19:08:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6032999999999998e-05 2021-01-15T19:08:19.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257171+0000", 2021-01-15T19:08:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003 2021-01-15T19:08:19.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.152 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=76, reqid=client.4418.0:184, at_version=27'9, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7830272639999998, 2021-01-15T19:08:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032503499999999999, 2021-01-15T19:08:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:19.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:19.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012968499999999999 2021-01-15T19:08:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254590+0000", 2021-01-15T19:08:19.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5418999999999999e-05 2021-01-15T19:08:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254605+0000", 2021-01-15T19:08:19.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999996e-06 2021-01-15T19:08:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254615+0000", 2021-01-15T19:08:19.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7790000000000003e-06 2021-01-15T19:08:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254619+0000", 2021-01-15T19:08:19.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3176000000000001e-05 2021-01-15T19:08:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254663+0000", 2021-01-15T19:08:19.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1525000000000001e-05 2021-01-15T19:08:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254674+0000", 2021-01-15T19:08:19.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030008919999999998 2021-01-15T19:08:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257675+0000", 2021-01-15T19:08:19.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5274000000000001e-05 2021-01-15T19:08:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257710+0000", 2021-01-15T19:08:19.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032503499999999999 2021-01-15T19:08:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7819390909999999, 2021-01-15T19:08:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:19.165 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:19.166 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=79, reqid=client.4418.0:185, at_version=27'10, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7807441320000001, 2021-01-15T19:08:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999, 2021-01-15T19:08:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5377000000000001e-05 2021-01-15T19:08:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256768+0000", 2021-01-15T19:08:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3387000000000001e-05 2021-01-15T19:08:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256782+0000", 2021-01-15T19:08:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6764999999999998e-05 2021-01-15T19:08:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256819+0000", 2021-01-15T19:08:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256826+0000", 2021-01-15T19:08:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9475999999999998e-05 2021-01-15T19:08:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256886+0000", 2021-01-15T19:08:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1929e-05 2021-01-15T19:08:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256898+0000", 2021-01-15T19:08:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055282589999999998 2021-01-15T19:08:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262426+0000", 2021-01-15T19:08:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2537999999999997e-05 2021-01-15T19:08:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262479+0000", 2021-01-15T19:08:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999 2021-01-15T19:08:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7792124509999998, 2021-01-15T19:08:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7782882419999999, 2021-01-15T19:08:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:19.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:19.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:19.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7780642790000001, 2021-01-15T19:08:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=81, reqid=client.4418.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=27'10))", 2021-01-15T19:08:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7744881719999999, 2021-01-15T19:08:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002, 2021-01-15T19:08:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4539999999999999e-05 2021-01-15T19:08:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263024+0000", 2021-01-15T19:08:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9600000000000006e-06 2021-01-15T19:08:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263032+0000", 2021-01-15T19:08:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2021-01-15T19:08:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263041+0000", 2021-01-15T19:08:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3239999999999996e-06 2021-01-15T19:08:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263046+0000", 2021-01-15T19:08:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4546999999999999e-05 2021-01-15T19:08:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263090+0000", 2021-01-15T19:08:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0596e-05 2021-01-15T19:08:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263101+0000", 2021-01-15T19:08:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052094810000000002 2021-01-15T19:08:19.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268310+0000", 2021-01-15T19:08:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4725000000000003e-05 2021-01-15T19:08:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268355+0000", 2021-01-15T19:08:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002 2021-01-15T19:08:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.248 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:19.249 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7742016939999998, 2021-01-15T19:08:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999, 2021-01-15T19:08:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3807000000000001e-05 2021-01-15T19:08:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263309+0000", 2021-01-15T19:08:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263317+0000", 2021-01-15T19:08:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2021-01-15T19:08:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263327+0000", 2021-01-15T19:08:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9009999999999997e-06 2021-01-15T19:08:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263334+0000", 2021-01-15T19:08:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052278899999999998 2021-01-15T19:08:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268562+0000", 2021-01-15T19:08:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4100000000000001e-05 2021-01-15T19:08:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268576+0000", 2021-01-15T19:08:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999 2021-01-15T19:08:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=30, last_complete=27'11, committed=1, applied=1))", 2021-01-15T19:08:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7735309849999998, 2021-01-15T19:08:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048998940000000001, 2021-01-15T19:08:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.260 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:19.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9382000000000001e-05 2021-01-15T19:08:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263986+0000", 2021-01-15T19:08:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.117e-06 2021-01-15T19:08:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263992+0000", 2021-01-15T19:08:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2400000000000007e-06 2021-01-15T19:08:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264000+0000", 2021-01-15T19:08:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2640000000000002e-06 2021-01-15T19:08:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264004+0000", 2021-01-15T19:08:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046074050000000002 2021-01-15T19:08:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268612+0000", 2021-01-15T19:08:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024448599999999998 2021-01-15T19:08:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268856+0000", 2021-01-15T19:08:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048998940000000001 2021-01-15T19:08:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.268190+0000", 2021-01-15T19:08:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7692967720000001, 2021-01-15T19:08:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066046200000000001, 2021-01-15T19:08:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268190+0000", 2021-01-15T19:08:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268190+0000", 2021-01-15T19:08:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3248999999999998e-05 2021-01-15T19:08:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268244+0000", 2021-01-15T19:08:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1276e-05 2021-01-15T19:08:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268255+0000", 2021-01-15T19:08:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6018000000000001e-05 2021-01-15T19:08:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268271+0000", 2021-01-15T19:08:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0809999999999994e-06 2021-01-15T19:08:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268279+0000", 2021-01-15T19:08:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000367828 2021-01-15T19:08:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268647+0000", 2021-01-15T19:08:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020400999999999999 2021-01-15T19:08:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268851+0000", 2021-01-15T19:08:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066046200000000001 2021-01-15T19:08:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=30, last_complete=27'11, committed=1, applied=1))", 2021-01-15T19:08:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.268327+0000", 2021-01-15T19:08:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7691601179999998, 2021-01-15T19:08:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072657099999999997, 2021-01-15T19:08:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268327+0000", 2021-01-15T19:08:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268327+0000", 2021-01-15T19:08:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9302999999999999e-05 2021-01-15T19:08:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268376+0000", 2021-01-15T19:08:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2259999999999999e-06 2021-01-15T19:08:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268384+0000", 2021-01-15T19:08:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2412000000000001e-05 2021-01-15T19:08:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268396+0000", 2021-01-15T19:08:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1930000000000003e-06 2021-01-15T19:08:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268400+0000", 2021-01-15T19:08:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047790799999999998 2021-01-15T19:08:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268878+0000", 2021-01-15T19:08:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017552900000000001 2021-01-15T19:08:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269054+0000", 2021-01-15T19:08:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072657099999999997 2021-01-15T19:08:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=31, last_complete=27'12, committed=1, applied=1))", 2021-01-15T19:08:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.270526+0000", 2021-01-15T19:08:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.7669614349999998, 2021-01-15T19:08:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000478746, 2021-01-15T19:08:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270526+0000", 2021-01-15T19:08:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270526+0000", 2021-01-15T19:08:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0468e-05 2021-01-15T19:08:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270556+0000", 2021-01-15T19:08:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0440000000000003e-06 2021-01-15T19:08:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270561+0000", 2021-01-15T19:08:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2199999999999998e-06 2021-01-15T19:08:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270570+0000", 2021-01-15T19:08:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0420000000000002e-06 2021-01-15T19:08:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270576+0000", 2021-01-15T19:08:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4993000000000002e-05 2021-01-15T19:08:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270611+0000", 2021-01-15T19:08:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039397899999999998 2021-01-15T19:08:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271004+0000", 2021-01-15T19:08:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000478746 2021-01-15T19:08:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.301 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:19.303 INFO:tasks.rados.rados.0.smithi001.stdout:82: done (1 left) 2021-01-15T19:08:19.304 INFO:tasks.rados.rados.0.smithi001.stdout:83: done (0 left) 2021-01-15T19:08:19.304 INFO:tasks.rados.rados.0.smithi001.stdout:84: snap_remove snap 3 2021-01-15T19:08:19.304 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:08:19.305 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:19.305 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:19.302+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:19.404 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:19.402+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:19.470 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:19.470 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:19.470 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:19.470 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:19.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:19.522 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:19.521+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:19.604 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:19.604 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:19.604 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:19.605 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:19.605 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:19.607 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:19.606+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:19.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:19.708 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:19.707+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:19.780 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=34, reqid=client.4418.0:156, at_version=26'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5343991790000002, 2021-01-15T19:08:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002, 2021-01-15T19:08:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166446 2021-01-15T19:08:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243528+0000", 2021-01-15T19:08:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5724e-05 2021-01-15T19:08:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243554+0000", 2021-01-15T19:08:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3528e-05 2021-01-15T19:08:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243567+0000", 2021-01-15T19:08:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4610000000000008e-06 2021-01-15T19:08:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243576+0000", 2021-01-15T19:08:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1644999999999999e-05 2021-01-15T19:08:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243618+0000", 2021-01-15T19:08:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2930000000000001e-06 2021-01-15T19:08:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243626+0000", 2021-01-15T19:08:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020585310000000002 2021-01-15T19:08:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245684+0000", 2021-01-15T19:08:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7432999999999997e-05 2021-01-15T19:08:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245742+0000", 2021-01-15T19:08:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002 2021-01-15T19:08:19.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.794 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=36, reqid=client.4418.0:157, at_version=26'13, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.531916678, 2021-01-15T19:08:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018949259999999999, 2021-01-15T19:08:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019949400000000001 2021-01-15T19:08:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246044+0000", 2021-01-15T19:08:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3130999999999999e-05 2021-01-15T19:08:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246067+0000", 2021-01-15T19:08:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8967999999999999e-05 2021-01-15T19:08:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246086+0000", 2021-01-15T19:08:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7729999999999999e-06 2021-01-15T19:08:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246094+0000", 2021-01-15T19:08:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3356999999999998e-05 2021-01-15T19:08:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246137+0000", 2021-01-15T19:08:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0242999999999999e-05 2021-01-15T19:08:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246157+0000", 2021-01-15T19:08:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015296000000000001 2021-01-15T19:08:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.247687+0000", 2021-01-15T19:08:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.236e-05 2021-01-15T19:08:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.247739+0000", 2021-01-15T19:08:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018949259999999999 2021-01-15T19:08:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=37, reqid=client.4418.0:158, at_version=26'14, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.254309+0000", 2021-01-15T19:08:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5234523519999996, 2021-01-15T19:08:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068320900000000005, 2021-01-15T19:08:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254309+0000", 2021-01-15T19:08:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254309+0000", 2021-01-15T19:08:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9445999999999999e-05 2021-01-15T19:08:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254338+0000", 2021-01-15T19:08:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0154000000000001e-05 2021-01-15T19:08:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254358+0000", 2021-01-15T19:08:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2536e-05 2021-01-15T19:08:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254378+0000", 2021-01-15T19:08:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7002000000000001e-05 2021-01-15T19:08:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254435+0000", 2021-01-15T19:08:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2788e-05 2021-01-15T19:08:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254447+0000", 2021-01-15T19:08:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049015799999999998 2021-01-15T19:08:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254937+0000", 2021-01-15T19:08:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4296000000000003e-05 2021-01-15T19:08:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.254992+0000", 2021-01-15T19:08:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068320900000000005 2021-01-15T19:08:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=39, reqid=client.4418.0:158, at_version=0'0, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5198471089999996, 2021-01-15T19:08:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001101858, 2021-01-15T19:08:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5347999999999999e-05 2021-01-15T19:08:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257969+0000", 2021-01-15T19:08:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5548000000000001e-05 2021-01-15T19:08:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257985+0000", 2021-01-15T19:08:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8144000000000001e-05 2021-01-15T19:08:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258003+0000", 2021-01-15T19:08:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2830000000000004e-06 2021-01-15T19:08:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258010+0000", 2021-01-15T19:08:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135764 2021-01-15T19:08:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258146+0000", 2021-01-15T19:08:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3592e-05 2021-01-15T19:08:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258159+0000", 2021-01-15T19:08:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080877599999999998 2021-01-15T19:08:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258968+0000", 2021-01-15T19:08:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7403e-05 2021-01-15T19:08:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259016+0000", 2021-01-15T19:08:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001101858 2021-01-15T19:08:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 26/21 ECSubWrite(tid=69, reqid=client.4418.0:173, at_version=26'11, trim_to=0'0, roll_forward_to=25'9))", 2021-01-15T19:08:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4991496800000004, 2021-01-15T19:08:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087352800000000004, 2021-01-15T19:08:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6239000000000002e-05 2021-01-15T19:08:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278657+0000", 2021-01-15T19:08:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2021-01-15T19:08:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278685+0000", 2021-01-15T19:08:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8410000000000002e-05 2021-01-15T19:08:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278704+0000", 2021-01-15T19:08:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550999999999999e-05 2021-01-15T19:08:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278714+0000", 2021-01-15T19:08:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0232999999999999e-05 2021-01-15T19:08:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278754+0000", 2021-01-15T19:08:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0690000000000008e-06 2021-01-15T19:08:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278764+0000", 2021-01-15T19:08:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067577100000000003 2021-01-15T19:08:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.279439+0000", 2021-01-15T19:08:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5459e-05 2021-01-15T19:08:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.279485+0000", 2021-01-15T19:08:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087352800000000004 2021-01-15T19:08:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 26/21 ECSubWrite(tid=70, reqid=client.4418.0:173, at_version=0'0, trim_to=0'0, roll_forward_to=26'11))", 2021-01-15T19:08:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.280242+0000", 2021-01-15T19:08:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4975191920000004, 2021-01-15T19:08:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078208300000000004, 2021-01-15T19:08:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280242+0000", 2021-01-15T19:08:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280242+0000", 2021-01-15T19:08:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:08:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280291+0000", 2021-01-15T19:08:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4776e-05 2021-01-15T19:08:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280306+0000", 2021-01-15T19:08:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8844000000000001e-05 2021-01-15T19:08:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280325+0000", 2021-01-15T19:08:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2719999999999997e-06 2021-01-15T19:08:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280333+0000", 2021-01-15T19:08:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102852 2021-01-15T19:08:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280436+0000", 2021-01-15T19:08:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.256e-05 2021-01-15T19:08:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280449+0000", 2021-01-15T19:08:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052864899999999996 2021-01-15T19:08:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.280977+0000", 2021-01-15T19:08:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6461000000000001e-05 2021-01-15T19:08:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281024+0000", 2021-01-15T19:08:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078208300000000004 2021-01-15T19:08:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=21, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4953997069999998, 2021-01-15T19:08:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010378760000000001, 2021-01-15T19:08:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4665e-05 2021-01-15T19:08:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282406+0000", 2021-01-15T19:08:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330999999999999e-05 2021-01-15T19:08:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282418+0000", 2021-01-15T19:08:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8114e-05 2021-01-15T19:08:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282436+0000", 2021-01-15T19:08:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4099999999999997e-06 2021-01-15T19:08:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282446+0000", 2021-01-15T19:08:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120245 2021-01-15T19:08:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282566+0000", 2021-01-15T19:08:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083311099999999999 2021-01-15T19:08:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283399+0000", 2021-01-15T19:08:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010378760000000001 2021-01-15T19:08:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=22, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4950471060000003, 2021-01-15T19:08:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090976900000000005, 2021-01-15T19:08:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9024999999999999e-05 2021-01-15T19:08:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282763+0000", 2021-01-15T19:08:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2038999999999999e-05 2021-01-15T19:08:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282775+0000", 2021-01-15T19:08:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9735999999999999e-05 2021-01-15T19:08:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282795+0000", 2021-01-15T19:08:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4930000000000003e-06 2021-01-15T19:08:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282801+0000", 2021-01-15T19:08:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063796299999999996 2021-01-15T19:08:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283439+0000", 2021-01-15T19:08:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184513 2021-01-15T19:08:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283624+0000", 2021-01-15T19:08:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090976900000000005 2021-01-15T19:08:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=74, reqid=client.4418.0:190, at_version=27'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5259280390000001, 2021-01-15T19:08:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002, 2021-01-15T19:08:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8488000000000001e-05 2021-01-15T19:08:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251871+0000", 2021-01-15T19:08:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0378e-05 2021-01-15T19:08:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251902+0000", 2021-01-15T19:08:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3121e-05 2021-01-15T19:08:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251915+0000", 2021-01-15T19:08:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3499999999999996e-06 2021-01-15T19:08:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251920+0000", 2021-01-15T19:08:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108286 2021-01-15T19:08:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252028+0000", 2021-01-15T19:08:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1652e-05 2021-01-15T19:08:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252040+0000", 2021-01-15T19:08:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002924991 2021-01-15T19:08:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254965+0000", 2021-01-15T19:08:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4570000000000001e-05 2021-01-15T19:08:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255020+0000", 2021-01-15T19:08:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002 2021-01-15T19:08:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=75, reqid=client.4418.0:191, at_version=27'13, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5244209099999999, 2021-01-15T19:08:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022390299999999999, 2021-01-15T19:08:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0649999999999998e-05 2021-01-15T19:08:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253391+0000", 2021-01-15T19:08:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0658000000000001e-05 2021-01-15T19:08:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253411+0000", 2021-01-15T19:08:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8773000000000001e-05 2021-01-15T19:08:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253430+0000", 2021-01-15T19:08:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4130000000000006e-06 2021-01-15T19:08:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253438+0000", 2021-01-15T19:08:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4926000000000003e-05 2021-01-15T19:08:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253483+0000", 2021-01-15T19:08:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3898e-05 2021-01-15T19:08:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253497+0000", 2021-01-15T19:08:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020547130000000001 2021-01-15T19:08:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255552+0000", 2021-01-15T19:08:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6999e-05 2021-01-15T19:08:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255579+0000", 2021-01-15T19:08:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022390299999999999 2021-01-15T19:08:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=78, reqid=client.4418.0:192, at_version=27'14, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256226+0000", 2021-01-15T19:08:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5215352129999999, 2021-01-15T19:08:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001558516, 2021-01-15T19:08:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256226+0000", 2021-01-15T19:08:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256226+0000", 2021-01-15T19:08:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0145999999999997e-05 2021-01-15T19:08:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256276+0000", 2021-01-15T19:08:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3802e-05 2021-01-15T19:08:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256300+0000", 2021-01-15T19:08:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8654000000000001e-05 2021-01-15T19:08:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256318+0000", 2021-01-15T19:08:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7539999999999995e-06 2021-01-15T19:08:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256327+0000", 2021-01-15T19:08:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2339999999999997e-05 2021-01-15T19:08:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256379+0000", 2021-01-15T19:08:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1408e-05 2021-01-15T19:08:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256391+0000", 2021-01-15T19:08:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001337792 2021-01-15T19:08:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257729+0000", 2021-01-15T19:08:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5619999999999999e-05 2021-01-15T19:08:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257784+0000", 2021-01-15T19:08:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001558516 2021-01-15T19:08:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=80, reqid=client.4418.0:192, at_version=0'0, trim_to=0'0, roll_forward_to=27'14))", 2021-01-15T19:08:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258427+0000", 2021-01-15T19:08:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5193343730000004, 2021-01-15T19:08:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088728699999999995, 2021-01-15T19:08:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258427+0000", 2021-01-15T19:08:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258427+0000", 2021-01-15T19:08:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6684999999999999e-05 2021-01-15T19:08:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258453+0000", 2021-01-15T19:08:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0017000000000001e-05 2021-01-15T19:08:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258463+0000", 2021-01-15T19:08:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0667e-05 2021-01-15T19:08:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258474+0000", 2021-01-15T19:08:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7099999999999998e-06 2021-01-15T19:08:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258479+0000", 2021-01-15T19:08:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0571999999999997e-05 2021-01-15T19:08:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258519+0000", 2021-01-15T19:08:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0327e-05 2021-01-15T19:08:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258529+0000", 2021-01-15T19:08:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075387700000000002 2021-01-15T19:08:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259283+0000", 2021-01-15T19:08:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0431999999999999e-05 2021-01-15T19:08:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259314+0000", 2021-01-15T19:08:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088728699999999995 2021-01-15T19:08:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.cs1 28/21 ECSubRead(tid=42, to_read={3:30718dd9:::smithi00140323-34:head=0,278528,0}, subchunks={3:30718dd9:::smithi00140323-34:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.251662+0000", 2021-01-15T19:08:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5260989089999999, 2021-01-15T19:08:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001207598, 2021-01-15T19:08:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251662+0000", 2021-01-15T19:08:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251662+0000", 2021-01-15T19:08:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.702e-05 2021-01-15T19:08:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251709+0000", 2021-01-15T19:08:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1551e-05 2021-01-15T19:08:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251721+0000", 2021-01-15T19:08:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9273999999999999e-05 2021-01-15T19:08:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251740+0000", 2021-01-15T19:08:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2250000000000006e-06 2021-01-15T19:08:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251749+0000", 2021-01-15T19:08:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5204999999999997e-05 2021-01-15T19:08:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251814+0000", 2021-01-15T19:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010553229999999999 2021-01-15T19:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.252870+0000", 2021-01-15T19:08:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001207598 2021-01-15T19:08:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.cs1 28/21 ECSubRead(tid=44, to_read={3:30718dd9:::smithi00140323-34:head=0,278528,0}, subchunks={3:30718dd9:::smithi00140323-34:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.254634+0000", 2021-01-15T19:08:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5231265629999999, 2021-01-15T19:08:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052635699999999998, 2021-01-15T19:08:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254634+0000", 2021-01-15T19:08:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254634+0000", 2021-01-15T19:08:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6419999999999999e-05 2021-01-15T19:08:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254681+0000", 2021-01-15T19:08:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0346999999999999e-05 2021-01-15T19:08:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254691+0000", 2021-01-15T19:08:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6432000000000001e-05 2021-01-15T19:08:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254708+0000", 2021-01-15T19:08:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2400000000000004e-06 2021-01-15T19:08:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254714+0000", 2021-01-15T19:08:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015751299999999999 2021-01-15T19:08:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.254871+0000", 2021-01-15T19:08:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028940500000000001 2021-01-15T19:08:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255161+0000", 2021-01-15T19:08:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052635699999999998 2021-01-15T19:08:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 28/21 ECSubWrite(tid=83, reqid=osd.4.0:83, at_version=28'16, trim_to=0'0, roll_forward_to=27'14))", 2021-01-15T19:08:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.255432+0000", 2021-01-15T19:08:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5223289750000002, 2021-01-15T19:08:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019518389999999999, 2021-01-15T19:08:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255432+0000", 2021-01-15T19:08:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255432+0000", 2021-01-15T19:08:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2109e-05 2021-01-15T19:08:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255464+0000", 2021-01-15T19:08:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6886999999999999e-05 2021-01-15T19:08:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255481+0000", 2021-01-15T19:08:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9709999999999993e-06 2021-01-15T19:08:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255490+0000", 2021-01-15T19:08:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6489999999999996e-06 2021-01-15T19:08:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255496+0000", 2021-01-15T19:08:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9673999999999998e-05 2021-01-15T19:08:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255555+0000", 2021-01-15T19:08:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3528e-05 2021-01-15T19:08:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255569+0000", 2021-01-15T19:08:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00175373 2021-01-15T19:08:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.257322+0000", 2021-01-15T19:08:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1291000000000007e-05 2021-01-15T19:08:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.257384+0000", 2021-01-15T19:08:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019518389999999999 2021-01-15T19:08:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=45, reqid=client.4418.0:204, at_version=28'16, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5223136529999999, 2021-01-15T19:08:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002, 2021-01-15T19:08:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1822999999999997e-05 2021-01-15T19:08:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255479+0000", 2021-01-15T19:08:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7513999999999999e-05 2021-01-15T19:08:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255497+0000", 2021-01-15T19:08:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3723000000000001e-05 2021-01-15T19:08:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255510+0000", 2021-01-15T19:08:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0780000000000003e-06 2021-01-15T19:08:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255515+0000", 2021-01-15T19:08:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2679999999999998e-05 2021-01-15T19:08:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255558+0000", 2021-01-15T19:08:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.695e-06 2021-01-15T19:08:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255566+0000", 2021-01-15T19:08:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028394660000000001 2021-01-15T19:08:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258405+0000", 2021-01-15T19:08:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9791999999999999e-05 2021-01-15T19:08:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258435+0000", 2021-01-15T19:08:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002 2021-01-15T19:08:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 28/21 ECSubWrite(tid=85, reqid=osd.4.0:83, at_version=0'0, trim_to=0'0, roll_forward_to=28'16))", 2021-01-15T19:08:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.259623+0000", 2021-01-15T19:08:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5181378969999999, 2021-01-15T19:08:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001392885, 2021-01-15T19:08:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259623+0000", 2021-01-15T19:08:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259623+0000", 2021-01-15T19:08:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5905999999999997e-05 2021-01-15T19:08:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259669+0000", 2021-01-15T19:08:19.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2064e-05 2021-01-15T19:08:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259681+0000", 2021-01-15T19:08:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1871e-05 2021-01-15T19:08:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259693+0000", 2021-01-15T19:08:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2021-01-15T19:08:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259698+0000", 2021-01-15T19:08:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8547000000000001e-05 2021-01-15T19:08:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259746+0000", 2021-01-15T19:08:19.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5950000000000004e-06 2021-01-15T19:08:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259753+0000", 2021-01-15T19:08:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012176019999999999 2021-01-15T19:08:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260971+0000", 2021-01-15T19:08:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5182999999999998e-05 2021-01-15T19:08:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.261016+0000", 2021-01-15T19:08:19.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001392885 2021-01-15T19:08:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=47, reqid=osd.0.0:47, at_version=28'18, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.260417+0000", 2021-01-15T19:08:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5173434060000002, 2021-01-15T19:08:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017245170000000001, 2021-01-15T19:08:19.982 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260417+0000", 2021-01-15T19:08:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260417+0000", 2021-01-15T19:08:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0448000000000001e-05 2021-01-15T19:08:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260448+0000", 2021-01-15T19:08:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9599e-05 2021-01-15T19:08:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260468+0000", 2021-01-15T19:08:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1601e-05 2021-01-15T19:08:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:19.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260479+0000", 2021-01-15T19:08:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9600000000000003e-06 2021-01-15T19:08:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:19.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260486+0000", 2021-01-15T19:08:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0080999999999998e-05 2021-01-15T19:08:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:19.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260526+0000", 2021-01-15T19:08:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9719999999999998e-06 2021-01-15T19:08:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:19.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260535+0000", 2021-01-15T19:08:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015556070000000001 2021-01-15T19:08:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:19.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.262091+0000", 2021-01-15T19:08:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1248999999999997e-05 2021-01-15T19:08:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:19.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.262142+0000", 2021-01-15T19:08:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017245170000000001 2021-01-15T19:08:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=48, reqid=osd.0.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=28'18))", 2021-01-15T19:08:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.265263+0000", 2021-01-15T19:08:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.5124983589999998, 2021-01-15T19:08:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096451299999999996, 2021-01-15T19:08:19.994 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265263+0000", 2021-01-15T19:08:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265263+0000", 2021-01-15T19:08:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9433000000000001e-05 2021-01-15T19:08:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265292+0000", 2021-01-15T19:08:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8470000000000008e-06 2021-01-15T19:08:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265302+0000", 2021-01-15T19:08:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0881e-05 2021-01-15T19:08:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:19.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265313+0000", 2021-01-15T19:08:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:08:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265319+0000", 2021-01-15T19:08:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5311e-05 2021-01-15T19:08:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265364+0000", 2021-01-15T19:08:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5469999999999999e-06 2021-01-15T19:08:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265372+0000", 2021-01-15T19:08:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081769500000000001 2021-01-15T19:08:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.266190+0000", 2021-01-15T19:08:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7329999999999997e-05 2021-01-15T19:08:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:20.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.266227+0000", 2021-01-15T19:08:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096451299999999996 2021-01-15T19:08:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.6s1 28/21 ECSubRead(tid=87, to_read={3:6feca5e1:::smithi00140323-9:head=0,69632,0}, subchunks={3:6feca5e1:::smithi00140323-9:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.311449+0000", 2021-01-15T19:08:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.466311996, 2021-01-15T19:08:20.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060185700000000002, 2021-01-15T19:08:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311449+0000", 2021-01-15T19:08:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311449+0000", 2021-01-15T19:08:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5249000000000002e-05 2021-01-15T19:08:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311474+0000", 2021-01-15T19:08:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3749999999999992e-06 2021-01-15T19:08:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311484+0000", 2021-01-15T19:08:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2643000000000001e-05 2021-01-15T19:08:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311496+0000", 2021-01-15T19:08:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6520000000000003e-06 2021-01-15T19:08:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311502+0000", 2021-01-15T19:08:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5637999999999999e-05 2021-01-15T19:08:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.311547+0000", 2021-01-15T19:08:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050330000000000004 2021-01-15T19:08:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:20.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.312051+0000", 2021-01-15T19:08:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060185700000000002 2021-01-15T19:08:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:20.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:20.016 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:20.016 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:20.017 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:20.016+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:20.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:20.119 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:20.117+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:20.163 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:20.163 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:20.163 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:20.163 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:20.173 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:20.219 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:20.217+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:20.263 INFO:tasks.rados.rados.0.smithi001.stdout:84: done (0 left) 2021-01-15T19:08:20.263 INFO:tasks.rados.rados.0.smithi001.stdout:85: snap_create 2021-01-15T19:08:20.307 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:20.308 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:20.308 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:20.308 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:20.308 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:20.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:20.320 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:20.318+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:20.420 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:20.419+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:20.450 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:20.451 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:08:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:154 3.fs0 3:f27a3046:::smithi00140323-1:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:20.452 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2106728029999996, 2021-01-15T19:08:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002, 2021-01-15T19:08:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:20.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 154 2021-01-15T19:08:20.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:20.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4352999999999999e-05 2021-01-15T19:08:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237631+0000", 2021-01-15T19:08:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3304e-05 2021-01-15T19:08:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237644+0000", 2021-01-15T19:08:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1736e-05 2021-01-15T19:08:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237666+0000", 2021-01-15T19:08:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0607e-05 2021-01-15T19:08:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237676+0000", 2021-01-15T19:08:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3862e-05 2021-01-15T19:08:20.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237730+0000", 2021-01-15T19:08:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1849999999999998e-06 2021-01-15T19:08:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:20.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237737+0000", 2021-01-15T19:08:20.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014276250000000001 2021-01-15T19:08:20.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239165+0000", 2021-01-15T19:08:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7447000000000007e-05 2021-01-15T19:08:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239232+0000", 2021-01-15T19:08:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061349480000000003 2021-01-15T19:08:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245367+0000", 2021-01-15T19:08:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095382000000000002 2021-01-15T19:08:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254906+0000", 2021-01-15T19:08:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113465 2021-01-15T19:08:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255019+0000", 2021-01-15T19:08:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000454659 2021-01-15T19:08:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255474+0000", 2021-01-15T19:08:20.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002 2021-01-15T19:08:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:155 3.fs0 3:f27a3046:::smithi00140323-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:20.473 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2105116950000001, 2021-01-15T19:08:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999, 2021-01-15T19:08:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:20.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 155 2021-01-15T19:08:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9607999999999998e-05 2021-01-15T19:08:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237767+0000", 2021-01-15T19:08:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3810000000000001e-06 2021-01-15T19:08:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237771+0000", 2021-01-15T19:08:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5639999999999999e-06 2021-01-15T19:08:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237779+0000", 2021-01-15T19:08:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.348e-06 2021-01-15T19:08:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237782+0000", 2021-01-15T19:08:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3377000000000002e-05 2021-01-15T19:08:20.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237816+0000", 2021-01-15T19:08:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000003e-06 2021-01-15T19:08:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237822+0000", 2021-01-15T19:08:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552276 2021-01-15T19:08:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239374+0000", 2021-01-15T19:08:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:20.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239415+0000", 2021-01-15T19:08:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010934268 2021-01-15T19:08:20.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250349+0000", 2021-01-15T19:08:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011671 2021-01-15T19:08:20.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250466+0000", 2021-01-15T19:08:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050409820000000003 2021-01-15T19:08:20.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255507+0000", 2021-01-15T19:08:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0531000000000005e-05 2021-01-15T19:08:20.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255587+0000", 2021-01-15T19:08:20.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122043 2021-01-15T19:08:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255709+0000", 2021-01-15T19:08:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999 2021-01-15T19:08:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:156 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~442368] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2066713360000003, 2021-01-15T19:08:20.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759, 2021-01-15T19:08:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156 2021-01-15T19:08:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030551200000000003 2021-01-15T19:08:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241873+0000", 2021-01-15T19:08:20.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6080000000000002e-06 2021-01-15T19:08:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241883+0000", 2021-01-15T19:08:20.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241898+0000", 2021-01-15T19:08:20.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4010000000000002e-06 2021-01-15T19:08:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241904+0000", 2021-01-15T19:08:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6971999999999997e-05 2021-01-15T19:08:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241951+0000", 2021-01-15T19:08:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104244 2021-01-15T19:08:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242055+0000", 2021-01-15T19:08:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070015099999999999 2021-01-15T19:08:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242755+0000", 2021-01-15T19:08:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0080604460000000006 2021-01-15T19:08:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250816+0000", 2021-01-15T19:08:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00338733 2021-01-15T19:08:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254203+0000", 2021-01-15T19:08:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3444999999999998e-05 2021-01-15T19:08:20.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254287+0000", 2021-01-15T19:08:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759 2021-01-15T19:08:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:157 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~401408] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2038651089999997, 2021-01-15T19:08:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335, 2021-01-15T19:08:20.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 157 2021-01-15T19:08:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000301899 2021-01-15T19:08:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244676+0000", 2021-01-15T19:08:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6430000000000001e-06 2021-01-15T19:08:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244685+0000", 2021-01-15T19:08:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5293000000000001e-05 2021-01-15T19:08:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244700+0000", 2021-01-15T19:08:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6499999999999996e-06 2021-01-15T19:08:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244708+0000", 2021-01-15T19:08:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1064999999999997e-05 2021-01-15T19:08:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244789+0000", 2021-01-15T19:08:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117805 2021-01-15T19:08:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244906+0000", 2021-01-15T19:08:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070339100000000004 2021-01-15T19:08:20.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245610+0000", 2021-01-15T19:08:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087607599999999994 2021-01-15T19:08:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107695 2021-01-15T19:08:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:20.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254478+0000", 2021-01-15T19:08:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109134 2021-01-15T19:08:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254587+0000", 2021-01-15T19:08:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335 2021-01-15T19:08:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:158 3.cs0 3:3ef45b66:::smithi00140323-11:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:20.530 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2034790920000003, 2021-01-15T19:08:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441, 2021-01-15T19:08:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:20.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 158 2021-01-15T19:08:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.532 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5699999999999999e-05 2021-01-15T19:08:20.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244816+0000", 2021-01-15T19:08:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6839999999999996e-06 2021-01-15T19:08:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244823+0000", 2021-01-15T19:08:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4626e-05 2021-01-15T19:08:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244838+0000", 2021-01-15T19:08:20.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6500000000000001e-06 2021-01-15T19:08:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244844+0000", 2021-01-15T19:08:20.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004205149 2021-01-15T19:08:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249049+0000", 2021-01-15T19:08:20.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015840599999999999 2021-01-15T19:08:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249207+0000", 2021-01-15T19:08:20.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071027700000000004 2021-01-15T19:08:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249918+0000", 2021-01-15T19:08:20.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051411900000000003 2021-01-15T19:08:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255059+0000", 2021-01-15T19:08:20.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045701699999999999 2021-01-15T19:08:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255516+0000", 2021-01-15T19:08:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8710999999999997e-05 2021-01-15T19:08:20.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255614+0000", 2021-01-15T19:08:20.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441 2021-01-15T19:08:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:159 3.cs0 3:3ef45b66:::smithi00140323-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2033466410000004, 2021-01-15T19:08:20.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195, 2021-01-15T19:08:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 159 2021-01-15T19:08:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0961000000000003e-05 2021-01-15T19:08:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244933+0000", 2021-01-15T19:08:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0480000000000001e-06 2021-01-15T19:08:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244941+0000", 2021-01-15T19:08:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3711e-05 2021-01-15T19:08:20.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244954+0000", 2021-01-15T19:08:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2390000000000002e-06 2021-01-15T19:08:20.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244959+0000", 2021-01-15T19:08:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082968859999999998 2021-01-15T19:08:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253256+0000", 2021-01-15T19:08:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000147608 2021-01-15T19:08:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253404+0000", 2021-01-15T19:08:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047189329999999998 2021-01-15T19:08:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258123+0000", 2021-01-15T19:08:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010107000000000001 2021-01-15T19:08:20.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258224+0000", 2021-01-15T19:08:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019277389999999999 2021-01-15T19:08:20.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260152+0000", 2021-01-15T19:08:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195 2021-01-15T19:08:20.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=34, last_complete=26'12, committed=1, applied=1))", 2021-01-15T19:08:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.245994+0000", 2021-01-15T19:08:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.2022452479999997, 2021-01-15T19:08:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047559530000000003, 2021-01-15T19:08:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.564 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245994+0000", 2021-01-15T19:08:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245994+0000", 2021-01-15T19:08:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5293000000000001e-05 2021-01-15T19:08:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246049+0000", 2021-01-15T19:08:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6680000000000007e-06 2021-01-15T19:08:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246058+0000", 2021-01-15T19:08:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3308e-05 2021-01-15T19:08:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246071+0000", 2021-01-15T19:08:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.798e-06 2021-01-15T19:08:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.246078+0000", 2021-01-15T19:08:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046489740000000002 2021-01-15T19:08:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250727+0000", 2021-01-15T19:08:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2912e-05 2021-01-15T19:08:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250750+0000", 2021-01-15T19:08:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047559530000000003 2021-01-15T19:08:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=34, last_complete=26'12, committed=1, applied=1))", 2021-01-15T19:08:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.250662+0000", 2021-01-15T19:08:20.574 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.1975775559999997, 2021-01-15T19:08:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036457899999999999, 2021-01-15T19:08:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250662+0000", 2021-01-15T19:08:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250662+0000", 2021-01-15T19:08:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003366815 2021-01-15T19:08:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254028+0000", 2021-01-15T19:08:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1878e-05 2021-01-15T19:08:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254040+0000", 2021-01-15T19:08:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4190000000000008e-06 2021-01-15T19:08:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254050+0000", 2021-01-15T19:08:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1590000000000003e-06 2021-01-15T19:08:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254054+0000", 2021-01-15T19:08:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3614e-05 2021-01-15T19:08:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254097+0000", 2021-01-15T19:08:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000209905 2021-01-15T19:08:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254307+0000", 2021-01-15T19:08:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036457899999999999 2021-01-15T19:08:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.2s2 26/21 ECSubWrite(tid=60, reqid=client.4418.0:166, at_version=26'8, trim_to=0'0, roll_forward_to=25'6))", 2021-01-15T19:08:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:20.588 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.1953965389999999, 2021-01-15T19:08:20.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053879080000000003, 2021-01-15T19:08:20.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.589 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:20.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:20.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000255274 2021-01-15T19:08:20.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253098+0000", 2021-01-15T19:08:20.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6628999999999997e-05 2021-01-15T19:08:20.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253145+0000", 2021-01-15T19:08:20.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7581e-05 2021-01-15T19:08:20.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253162+0000", 2021-01-15T19:08:20.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3610000000000003e-06 2021-01-15T19:08:20.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253169+0000", 2021-01-15T19:08:20.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2859000000000003e-05 2021-01-15T19:08:20.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253222+0000", 2021-01-15T19:08:20.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2021000000000001e-05 2021-01-15T19:08:20.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253234+0000", 2021-01-15T19:08:20.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049458130000000003 2021-01-15T19:08:20.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258180+0000", 2021-01-15T19:08:20.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0370000000000001e-05 2021-01-15T19:08:20.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258231+0000", 2021-01-15T19:08:20.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053879080000000003 2021-01-15T19:08:20.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.601 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 26/21 ECSubWriteReply(tid=37, last_complete=26'14, committed=1, applied=1))", 2021-01-15T19:08:20.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.255288+0000", 2021-01-15T19:08:20.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.1929514409999999, 2021-01-15T19:08:20.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041825289999999999, 2021-01-15T19:08:20.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.602 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255288+0000", 2021-01-15T19:08:20.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255288+0000", 2021-01-15T19:08:20.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3288999999999998e-05 2021-01-15T19:08:20.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255341+0000", 2021-01-15T19:08:20.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1599999999999998e-06 2021-01-15T19:08:20.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255349+0000", 2021-01-15T19:08:20.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2616000000000001e-05 2021-01-15T19:08:20.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255362+0000", 2021-01-15T19:08:20.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4280000000000004e-06 2021-01-15T19:08:20.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255367+0000", 2021-01-15T19:08:20.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3349000000000001e-05 2021-01-15T19:08:20.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255411+0000", 2021-01-15T19:08:20.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040596870000000002 2021-01-15T19:08:20.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.259470+0000", 2021-01-15T19:08:20.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041825289999999999 2021-01-15T19:08:20.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.611 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:201 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:20.611 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:20.611 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1999933140000003, 2021-01-15T19:08:20.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142, 2021-01-15T19:08:20.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:20.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 201 2021-01-15T19:08:20.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.613 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:20.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:20.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1060000000000002e-05 2021-01-15T19:08:20.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248307+0000", 2021-01-15T19:08:20.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6740000000000003e-06 2021-01-15T19:08:20.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248316+0000", 2021-01-15T19:08:20.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8746e-05 2021-01-15T19:08:20.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248334+0000", 2021-01-15T19:08:20.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2294e-05 2021-01-15T19:08:20.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248347+0000", 2021-01-15T19:08:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6400999999999998e-05 2021-01-15T19:08:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248393+0000", 2021-01-15T19:08:20.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2021-01-15T19:08:20.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:20.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248400+0000", 2021-01-15T19:08:20.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002761197 2021-01-15T19:08:20.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251161+0000", 2021-01-15T19:08:20.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011450099999999999 2021-01-15T19:08:20.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251275+0000", 2021-01-15T19:08:20.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062315829999999997 2021-01-15T19:08:20.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257507+0000", 2021-01-15T19:08:20.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142 2021-01-15T19:08:20.624 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.624 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.625 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:202 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:20.625 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:20.626 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1998399390000003, 2021-01-15T19:08:20.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285, 2021-01-15T19:08:20.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:20.627 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.627 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.627 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 202 2021-01-15T19:08:20.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.628 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:20.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:20.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5448999999999997e-05 2021-01-15T19:08:20.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248455+0000", 2021-01-15T19:08:20.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:20.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248461+0000", 2021-01-15T19:08:20.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4307e-05 2021-01-15T19:08:20.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248476+0000", 2021-01-15T19:08:20.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5819999999999998e-06 2021-01-15T19:08:20.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248482+0000", 2021-01-15T19:08:20.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1987e-05 2021-01-15T19:08:20.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248564+0000", 2021-01-15T19:08:20.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5850000000000002e-06 2021-01-15T19:08:20.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:20.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248572+0000", 2021-01-15T19:08:20.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045194099999999997 2021-01-15T19:08:20.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253091+0000", 2021-01-15T19:08:20.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020063399999999999 2021-01-15T19:08:20.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253292+0000", 2021-01-15T19:08:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079735020000000004 2021-01-15T19:08:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261266+0000", 2021-01-15T19:08:20.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285 2021-01-15T19:08:20.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.639 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.640 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:203 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:20.640 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:20.640 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1996609210000004, 2021-01-15T19:08:20.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438, 2021-01-15T19:08:20.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:20.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 203 2021-01-15T19:08:20.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:20.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:20.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8518999999999999e-05 2021-01-15T19:08:20.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248607+0000", 2021-01-15T19:08:20.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8810000000000004e-06 2021-01-15T19:08:20.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248612+0000", 2021-01-15T19:08:20.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5719999999999997e-06 2021-01-15T19:08:20.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248619+0000", 2021-01-15T19:08:20.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5899999999999999e-06 2021-01-15T19:08:20.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248623+0000", 2021-01-15T19:08:20.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5608000000000001e-05 2021-01-15T19:08:20.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248668+0000", 2021-01-15T19:08:20.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0166000000000001e-05 2021-01-15T19:08:20.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:20.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248679+0000", 2021-01-15T19:08:20.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048830590000000004 2021-01-15T19:08:20.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253562+0000", 2021-01-15T19:08:20.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7e-05 2021-01-15T19:08:20.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253659+0000", 2021-01-15T19:08:20.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081539850000000007 2021-01-15T19:08:20.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261813+0000", 2021-01-15T19:08:20.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438 2021-01-15T19:08:20.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:204 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat,delete] snapc 3=[3] ondisk+write+known_if_redirected e28)", 2021-01-15T19:08:20.655 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:20.655 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.199576714, 2021-01-15T19:08:20.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532, 2021-01-15T19:08:20.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:20.656 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:20.656 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:20.656 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:20.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 204 2021-01-15T19:08:20.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.657 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:20.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:20.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8349000000000002e-05 2021-01-15T19:08:20.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248701+0000", 2021-01-15T19:08:20.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0620000000000001e-06 2021-01-15T19:08:20.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248709+0000", 2021-01-15T19:08:20.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7700000000000001e-06 2021-01-15T19:08:20.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248717+0000", 2021-01-15T19:08:20.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5920000000000001e-06 2021-01-15T19:08:20.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248720+0000", 2021-01-15T19:08:20.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0898e-05 2021-01-15T19:08:20.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248761+0000", 2021-01-15T19:08:20.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-06 2021-01-15T19:08:20.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:20.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248770+0000", 2021-01-15T19:08:20.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052185720000000003 2021-01-15T19:08:20.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253989+0000", 2021-01-15T19:08:20.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105809 2021-01-15T19:08:20.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:20.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254095+0000", 2021-01-15T19:08:20.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078837199999999997 2021-01-15T19:08:20.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:20.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254883+0000", 2021-01-15T19:08:20.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083722769999999992 2021-01-15T19:08:20.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:20.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263255+0000", 2021-01-15T19:08:20.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111839 2021-01-15T19:08:20.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:20.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263367+0000", 2021-01-15T19:08:20.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015969899999999999 2021-01-15T19:08:20.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263527+0000", 2021-01-15T19:08:20.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532 2021-01-15T19:08:20.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.671 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.672 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpReadReply(3.cs0 28/21 ECSubReadReply(tid=42, attrs_read=0))", 2021-01-15T19:08:20.672 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.253041+0000", 2021-01-15T19:08:20.672 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1951979770000003, 2021-01-15T19:08:20.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037037369999999999, 2021-01-15T19:08:20.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.673 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253041+0000", 2021-01-15T19:08:20.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253041+0000", 2021-01-15T19:08:20.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253041+0000", 2021-01-15T19:08:20.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253041+0000", 2021-01-15T19:08:20.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00089334199999999999 2021-01-15T19:08:20.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253935+0000", 2021-01-15T19:08:20.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3600000000000004e-06 2021-01-15T19:08:20.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253940+0000", 2021-01-15T19:08:20.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027533610000000002 2021-01-15T19:08:20.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.256693+0000", 2021-01-15T19:08:20.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1674e-05 2021-01-15T19:08:20.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.256745+0000", 2021-01-15T19:08:20.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037037369999999999 2021-01-15T19:08:20.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.682 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.682 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpRead(3.cs0 28/21 ECSubRead(tid=43, to_read={3:30718dd9:::smithi00140323-34:head=0,278528,0}, subchunks={3:30718dd9:::smithi00140323-34:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:20.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.253458+0000", 2021-01-15T19:08:20.683 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1947810900000002, 2021-01-15T19:08:20.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003549932, 2021-01-15T19:08:20.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.683 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253458+0000", 2021-01-15T19:08:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253458+0000", 2021-01-15T19:08:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253458+0000", 2021-01-15T19:08:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253458+0000", 2021-01-15T19:08:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049862799999999998 2021-01-15T19:08:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253957+0000", 2021-01-15T19:08:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5210000000000002e-06 2021-01-15T19:08:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253962+0000", 2021-01-15T19:08:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028143719999999999 2021-01-15T19:08:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.256777+0000", 2021-01-15T19:08:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000231411 2021-01-15T19:08:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257008+0000", 2021-01-15T19:08:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003549932 2021-01-15T19:08:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpRead(3.cs0 28/21 ECSubRead(tid=44, to_read={3:30718dd9:::smithi00140323-34:head=0,278528,0}, subchunks={3:30718dd9:::smithi00140323-34:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.253820+0000", 2021-01-15T19:08:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1944195009999996, 2021-01-15T19:08:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039590659999999998, 2021-01-15T19:08:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253820+0000", 2021-01-15T19:08:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253820+0000", 2021-01-15T19:08:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253820+0000", 2021-01-15T19:08:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253820+0000", 2021-01-15T19:08:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018301999999999999 2021-01-15T19:08:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254003+0000", 2021-01-15T19:08:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8600000000000001e-05 2021-01-15T19:08:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254021+0000", 2021-01-15T19:08:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035369820000000001 2021-01-15T19:08:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257558+0000", 2021-01-15T19:08:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220464 2021-01-15T19:08:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257779+0000", 2021-01-15T19:08:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039590659999999998 2021-01-15T19:08:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpReadReply(3.cs0 28/21 ECSubReadReply(tid=42, attrs_read=0))", 2021-01-15T19:08:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.253958+0000", 2021-01-15T19:08:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1942809609999996, 2021-01-15T19:08:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035661080000000001, 2021-01-15T19:08:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253958+0000", 2021-01-15T19:08:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253958+0000", 2021-01-15T19:08:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265207 2021-01-15T19:08:20.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254223+0000", 2021-01-15T19:08:20.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.502e-05 2021-01-15T19:08:20.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254238+0000", 2021-01-15T19:08:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7003e-05 2021-01-15T19:08:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254255+0000", 2021-01-15T19:08:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011247399999999999 2021-01-15T19:08:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254368+0000", 2021-01-15T19:08:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002670689 2021-01-15T19:08:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257039+0000", 2021-01-15T19:08:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00048571500000000001 2021-01-15T19:08:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257524+0000", 2021-01-15T19:08:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035661080000000001 2021-01-15T19:08:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpReadReply(3.cs0 28/21 ECSubReadReply(tid=43, attrs_read=0))", 2021-01-15T19:08:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1912377989999996, 2021-01-15T19:08:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065310519999999999, 2021-01-15T19:08:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001039442 2021-01-15T19:08:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.258041+0000", 2021-01-15T19:08:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.784e-06 2021-01-15T19:08:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.258047+0000", 2021-01-15T19:08:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026103659999999998 2021-01-15T19:08:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.260657+0000", 2021-01-15T19:08:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028754599999999998 2021-01-15T19:08:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263532+0000", 2021-01-15T19:08:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065310519999999999 2021-01-15T19:08:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpReadReply(3.cs0 28/21 ECSubReadReply(tid=44, attrs_read=0))", 2021-01-15T19:08:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.257777+0000", 2021-01-15T19:08:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.1904625019999999, 2021-01-15T19:08:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040505080000000004, 2021-01-15T19:08:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257777+0000", 2021-01-15T19:08:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257777+0000", 2021-01-15T19:08:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:20.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257777+0000", 2021-01-15T19:08:20.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:20.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257777+0000", 2021-01-15T19:08:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00028445100000000002 2021-01-15T19:08:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.258061+0000", 2021-01-15T19:08:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6959999999999998e-06 2021-01-15T19:08:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.258065+0000", 2021-01-15T19:08:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033124719999999999 2021-01-15T19:08:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261377+0000", 2021-01-15T19:08:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044988899999999997 2021-01-15T19:08:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261827+0000", 2021-01-15T19:08:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040505080000000004 2021-01-15T19:08:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:20.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:20.733 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:20.735 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:20.733+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:20.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:20.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:20.836 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:20.835+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:20.912 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:20.912 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:20.922 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:20.925 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:20.938 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:20.937+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:20.946 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:20.967 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:20.988 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:21.010 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:21.033 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:21.039 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:21.038+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:21.054 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:21.055 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:21.055 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:21.066 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:21.076 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:21.141 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:21.139+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:21.208 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:21.208 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:21.218 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:21.258 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:21.257+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:21.262 INFO:tasks.rados.rados.0.smithi001.stdout:85: done (0 left) 2021-01-15T19:08:21.262 INFO:tasks.rados.rados.0.smithi001.stdout:86: snap_remove snap 5 2021-01-15T19:08:21.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:08:21.343 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:21.342+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:21.392 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:21.392 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:21.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:21.445 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:21.443+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:21.536 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:21.537 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:21.544 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:21.543+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:21.546 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:21.647 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:21.646+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:21.723 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:21.724 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:21.729 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:21.748 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:21.746+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:21.853 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:21.847+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:21.863 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:21.864 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:21.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:21.957 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:21.956+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:22.046 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:22.046 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:22.050 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:22.050+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:22.055 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:22.152 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:22.150+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:22.226 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:22.227 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:22.236 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:22.269 INFO:tasks.rados.rados.0.smithi001.stdout:86: done (0 left) 2021-01-15T19:08:22.269 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 33 current snap is 6 2021-01-15T19:08:22.270 INFO:tasks.rados.rados.0.smithi001.stdout:87: seq_num 58 ranges {0=573440,573440=401408} 2021-01-15T19:08:22.272 INFO:tasks.rados.rados.0.smithi001.stdout:87: writing smithi00140323-33 from 0 to 573440 tid 1 2021-01-15T19:08:22.273 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:08:22.275 INFO:tasks.rados.rados.0.smithi001.stdout:87: writing smithi00140323-33 from 573440 to 974848 tid 2 2021-01-15T19:08:22.275 INFO:tasks.rados.rados.0.smithi001.stdout:88: read oid 44 snap -1 2021-01-15T19:08:22.275 INFO:tasks.rados.rados.0.smithi001.stdout:88: expect (ObjNum 43 snap 0 seq_num 43) 2021-01-15T19:08:22.275 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 27 current snap is 6 2021-01-15T19:08:22.276 INFO:tasks.rados.rados.0.smithi001.stdout:89: seq_num 59 ranges {442368=598016,1040384=401408} 2021-01-15T19:08:22.281 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:22.280+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:22.282 INFO:tasks.rados.rados.0.smithi001.stdout:89: writing smithi00140323-27 from 442368 to 1040384 tid 1 2021-01-15T19:08:22.284 INFO:tasks.rados.rados.0.smithi001.stdout:89: writing smithi00140323-27 from 1040384 to 1441792 tid 2 2021-01-15T19:08:22.284 INFO:tasks.rados.rados.0.smithi001.stdout:90: snap_remove snap 2 2021-01-15T19:08:22.353 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:22.352+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:22.381 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:22.382 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:22.391 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:22.454 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:22.453+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:22.556 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:22.556 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:22.557 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:22.554+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:22.565 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:22.657 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:22.655+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:22.729 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:22.730 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:22.739 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:22.758 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:22.756+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:22.858 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:22.857+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:22.905 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:22.905 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:22.915 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:22.926 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:22.926 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:22.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2021-01-15T19:08:22.970 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:22.969+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:23.040 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:23.041 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:23.051 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:23.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:23.095 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:23.094+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:23.160 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:23.159+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:23.178 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:23.179 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:23.189 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:23.242 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:23.262 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 34, 2021-01-15T19:08:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "fe0e1890-043f-40bf-ad11-c5f08349f418", 2021-01-15T19:08:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2021-01-15T19:07:43.378003+0000", 2021-01-15T19:08:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2021-01-15T19:08:22.263011+0000", 2021-01-15T19:08:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2021-01-15T19:07:50.512119+0000", 2021-01-15T19:08:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2021-01-15T19:07:50.512119+0000", 2021-01-15T19:08:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-01-15T19:08:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5806080, 2021-01-15T19:08:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2021-01-15T19:08:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "nodeep-scrub", 2021-01-15T19:08:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2021-01-15T19:08:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2021-01-15T19:08:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2021-01-15T19:08:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2021-01-15T19:08:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2021-01-15T19:08:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:08:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2021-01-15T19:08:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2021-01-15T19:08:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-01-15T19:08:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-01-15T19:08:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2021-01-15T19:08:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2021-01-15T19:08:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2021-01-15T19:08:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2021-01-15T19:08:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "jewel", 2021-01-15T19:08:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "pacific", 2021-01-15T19:08:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2021-01-15T19:08:23.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.269 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2021-01-15T19:08:23.269 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "device_health_metrics", 2021-01-15T19:08:23.269 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:47.545867+0000", 2021-01-15T19:08:23.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:08:23.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:08:23.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:08:23.271 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:08:23.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:08:23.271 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:08:23.271 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:08:23.271 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:08:23.272 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:08:23.272 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:08:23.272 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:08:23.272 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:08:23.272 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 1, 2021-01-15T19:08:23.273 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 1, 2021-01-15T19:08:23.273 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 1, 2021-01-15T19:08:23.273 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 1, 2021-01-15T19:08:23.273 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 1, 2021-01-15T19:08:23.273 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:08:23.274 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:08:23.274 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:08:23.274 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:08:23.275 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:08:23.275 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:08:23.275 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:08:23.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.276 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "15", 2021-01-15T19:08:23.276 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:08:23.276 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:08:23.276 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2021-01-15T19:08:23.276 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:08:23.277 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:08:23.277 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:08:23.277 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:08:23.277 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:08:23.277 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:08:23.278 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:08:23.278 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:08:23.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:08:23.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:08:23.279 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:08:23.279 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:08:23.279 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:08:23.279 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:08:23.279 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:08:23.280 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:08:23.280 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:08:23.280 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:08:23.280 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:08:23.281 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:08:23.281 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:08:23.281 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:08:23.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:08:23.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.282 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:08:23.282 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:08:23.283 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:08:23.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:08:23.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:08:23.283 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:08:23.283 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:08:23.284 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:08:23.284 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:08:23.284 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:08:23.285 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:08:23.285 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2021-01-15T19:08:23.285 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2021-01-15T19:08:23.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.285 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:08:23.286 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_devicehealth": {} 2021-01-15T19:08:23.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.287 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2021-01-15T19:08:23.287 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2021-01-15T19:08:23.287 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:54.484638+0000", 2021-01-15T19:08:23.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:08:23.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:08:23.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:08:23.288 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:08:23.288 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:08:23.288 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:08:23.288 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:08:23.289 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:08:23.289 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:08:23.289 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:08:23.289 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:08:23.289 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:08:23.290 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 8, 2021-01-15T19:08:23.290 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 8, 2021-01-15T19:08:23.290 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 8, 2021-01-15T19:08:23.291 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 8, 2021-01-15T19:08:23.291 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 8, 2021-01-15T19:08:23.291 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:08:23.291 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:08:23.291 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:08:23.292 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:08:23.292 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:08:23.292 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:08:23.292 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:08:23.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.293 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "18", 2021-01-15T19:08:23.293 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:08:23.293 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:08:23.293 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2021-01-15T19:08:23.293 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:08:23.294 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:08:23.294 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:08:23.294 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:08:23.294 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:08:23.295 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:08:23.295 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:08:23.295 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:08:23.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:08:23.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:08:23.296 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:08:23.296 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:08:23.297 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:08:23.297 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:08:23.297 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:08:23.298 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:08:23.298 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:08:23.299 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:08:23.299 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:08:23.300 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:08:23.300 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:08:23.300 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:08:23.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:08:23.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.301 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:08:23.301 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:08:23.302 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:08:23.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:08:23.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:08:23.303 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:08:23.303 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:08:23.303 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:08:23.303 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:08:23.303 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:08:23.304 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:08:23.304 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:08:23.304 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:08:23.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2021-01-15T19:08:23.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.306 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:08:23.306 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "unique_pool_0", 2021-01-15T19:08:23.306 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:08:08.242953+0000", 2021-01-15T19:08:23.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2021-01-15T19:08:23.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2021-01-15T19:08:23.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 3, 2021-01-15T19:08:23.307 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 3, 2021-01-15T19:08:23.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 2, 2021-01-15T19:08:23.307 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 1, 2021-01-15T19:08:23.307 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:08:23.308 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:08:23.308 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:08:23.308 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:08:23.308 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:08:23.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:08:23.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 16, 2021-01-15T19:08:23.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 16, 2021-01-15T19:08:23.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 16, 2021-01-15T19:08:23.309 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 16, 2021-01-15T19:08:23.310 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 16, 2021-01-15T19:08:23.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:08:23.310 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:08:23.310 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:08:23.310 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:08:23.311 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:08:23.311 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:08:23.311 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:08:23.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "34", 2021-01-15T19:08:23.312 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:08:23.313 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:08:23.313 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "0", 2021-01-15T19:08:23.313 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:08:23.314 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:08:23.314 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 10, 2021-01-15T19:08:23.314 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 34, 2021-01-15T19:08:23.314 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:08:23.315 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:08:23.315 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:08:23.315 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:08:23.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:08:23.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:08:23.316 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:08:23.316 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:08:23.316 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:08:23.316 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:08:23.317 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:08:23.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:08:23.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:08:23.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:08:23.317 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:08:23.318 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:08:23.318 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "isaprofile", 2021-01-15T19:08:23.318 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:08:23.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:08:23.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.319 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:08:23.319 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:08:23.319 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:08:23.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:08:23.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:08:23.320 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:08:23.320 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:08:23.321 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:08:23.321 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 8192, 2021-01-15T19:08:23.321 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:08:23.321 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:08:23.321 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:08:23.322 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:08:23.322 INFO:teuthology.orchestra.run.smithi001.stdout: "rados": {} 2021-01-15T19:08:23.322 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.322 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.323 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:08:23.323 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:08:23.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.323 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:08:23.323 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e5b26f89-5c99-4764-9183-f3bb27c118f5", 2021-01-15T19:08:23.324 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.324 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.324 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.324 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.325 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.325 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.325 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.325 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.325 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.326 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.326 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.326 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.327 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2021-01-15T19:08:23.327 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.328 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2021-01-15T19:08:23.328 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.328 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.329 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.329 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.329 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.330 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2021-01-15T19:08:23.330 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2021-01-15T19:08:23.331 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.331 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.332 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.332 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.332 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.333 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2021-01-15T19:08:23.333 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.334 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2021-01-15T19:08:23.334 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2021-01-15T19:08:23.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2021-01-15T19:08:23.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:08:23.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.338 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/34833", 2021-01-15T19:08:23.338 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/34833", 2021-01-15T19:08:23.339 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/34833", 2021-01-15T19:08:23.339 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/34833", 2021-01-15T19:08:23.339 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.339 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.339 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.340 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.340 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:08:23.341 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "99c0d455-1bd6-43aa-bfec-b6bddf87a35e", 2021-01-15T19:08:23.341 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.341 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.341 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.342 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.342 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.342 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.342 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.342 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 13, 2021-01-15T19:08:23.343 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.343 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.343 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2021-01-15T19:08:23.344 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.345 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2021-01-15T19:08:23.345 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.346 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.346 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2021-01-15T19:08:23.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.348 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2021-01-15T19:08:23.349 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.349 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.349 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.349 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.350 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2021-01-15T19:08:23.351 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2021-01-15T19:08:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.353 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.353 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.354 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2021-01-15T19:08:23.354 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.355 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2021-01-15T19:08:23.355 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:08:23.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.356 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/34834", 2021-01-15T19:08:23.356 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/34834", 2021-01-15T19:08:23.356 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/34834", 2021-01-15T19:08:23.356 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/34834", 2021-01-15T19:08:23.357 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.357 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.357 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.358 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:08:23.358 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "71fe6c4b-3d73-4e07-b49d-98596ca7d9da", 2021-01-15T19:08:23.358 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.359 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.359 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.359 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.359 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.360 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.360 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.360 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.360 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.360 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.361 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2021-01-15T19:08:23.362 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.363 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2021-01-15T19:08:23.363 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.364 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.364 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.365 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2021-01-15T19:08:23.365 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.366 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2021-01-15T19:08:23.366 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.366 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.366 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.367 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2021-01-15T19:08:23.368 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2021-01-15T19:08:23.369 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.370 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.370 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.371 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2021-01-15T19:08:23.371 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.372 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2021-01-15T19:08:23.372 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:08:23.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.373 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.373 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6819/34863", 2021-01-15T19:08:23.373 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6823/34863", 2021-01-15T19:08:23.374 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/34863", 2021-01-15T19:08:23.374 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/34863", 2021-01-15T19:08:23.374 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.374 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.375 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.375 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:08:23.376 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8a6b018a-3a48-4a01-8526-a386f62ecb36", 2021-01-15T19:08:23.376 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.376 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.376 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.376 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.377 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.377 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.377 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.377 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.378 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.378 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.378 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.378 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.379 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2021-01-15T19:08:23.379 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.380 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2021-01-15T19:08:23.380 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.381 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.381 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.381 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.382 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2021-01-15T19:08:23.382 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.383 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2021-01-15T19:08:23.384 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.384 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.385 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.385 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2021-01-15T19:08:23.386 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.387 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2021-01-15T19:08:23.387 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.387 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.388 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.388 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.389 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2021-01-15T19:08:23.389 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.390 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2021-01-15T19:08:23.390 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:08:23.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.390 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.391 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6818/34864", 2021-01-15T19:08:23.391 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6822/34864", 2021-01-15T19:08:23.391 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/34864", 2021-01-15T19:08:23.392 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/34864", 2021-01-15T19:08:23.392 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.392 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.392 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.393 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:08:23.393 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "ff089622-e642-4bd0-901c-60efdb87a549", 2021-01-15T19:08:23.394 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.394 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.394 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.394 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.394 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.395 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.395 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.395 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.395 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.395 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.396 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.396 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.397 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6800", 2021-01-15T19:08:23.397 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.398 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6801", 2021-01-15T19:08:23.398 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.399 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.399 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.400 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6802", 2021-01-15T19:08:23.400 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.401 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6803", 2021-01-15T19:08:23.401 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.401 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.402 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.402 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.403 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6806", 2021-01-15T19:08:23.403 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.404 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6807", 2021-01-15T19:08:23.404 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.405 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.405 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.405 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.406 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6804", 2021-01-15T19:08:23.406 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.407 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6805", 2021-01-15T19:08:23.407 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:08:23.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.408 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6801/34829", 2021-01-15T19:08:23.408 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6803/34829", 2021-01-15T19:08:23.408 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6807/34829", 2021-01-15T19:08:23.409 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6805/34829", 2021-01-15T19:08:23.409 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.409 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.409 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.410 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.410 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:08:23.411 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "3c57c228-dfaa-4e33-bded-27da0048d141", 2021-01-15T19:08:23.411 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.411 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.411 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.411 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.412 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.412 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.412 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.412 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.412 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.413 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.413 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.413 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.414 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6809", 2021-01-15T19:08:23.414 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.415 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6811", 2021-01-15T19:08:23.415 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.416 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.416 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.416 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.417 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6813", 2021-01-15T19:08:23.417 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.418 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6815", 2021-01-15T19:08:23.418 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.419 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.419 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.419 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.420 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6821", 2021-01-15T19:08:23.420 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.421 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6823", 2021-01-15T19:08:23.421 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.422 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.422 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.422 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.423 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6817", 2021-01-15T19:08:23.423 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.424 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6819", 2021-01-15T19:08:23.424 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:08:23.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.425 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6811/34830", 2021-01-15T19:08:23.426 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6815/34830", 2021-01-15T19:08:23.426 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6823/34830", 2021-01-15T19:08:23.426 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6819/34830", 2021-01-15T19:08:23.426 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.426 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.427 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.428 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:08:23.428 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "201f889f-dac1-4a26-a285-7a0ec21c3646", 2021-01-15T19:08:23.428 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.428 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.429 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.429 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.429 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.429 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.429 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.430 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.430 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.430 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.430 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.431 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.431 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6808", 2021-01-15T19:08:23.431 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.432 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6810", 2021-01-15T19:08:23.433 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.433 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.433 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.433 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.434 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.434 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6812", 2021-01-15T19:08:23.435 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.436 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6814", 2021-01-15T19:08:23.436 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.436 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.437 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.437 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.438 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6820", 2021-01-15T19:08:23.438 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.439 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6822", 2021-01-15T19:08:23.439 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.439 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.440 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.440 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.441 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6816", 2021-01-15T19:08:23.441 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.442 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6818", 2021-01-15T19:08:23.442 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:08:23.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.443 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.443 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6810/34831", 2021-01-15T19:08:23.443 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6814/34831", 2021-01-15T19:08:23.443 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6822/34831", 2021-01-15T19:08:23.444 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6818/34831", 2021-01-15T19:08:23.444 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.444 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.444 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.444 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.445 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:08:23.445 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "79ffd413-a9d3-4180-9691-d834d89c3f2e", 2021-01-15T19:08:23.446 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:08:23.446 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:08:23.446 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:08:23.446 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:08:23.446 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:08:23.447 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:08:23.447 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:08:23.447 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 21, 2021-01-15T19:08:23.447 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:08:23.448 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:08:23.448 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:08:23.448 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.449 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6824", 2021-01-15T19:08:23.449 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.450 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6825", 2021-01-15T19:08:23.450 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.451 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:08:23.451 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.452 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6826", 2021-01-15T19:08:23.452 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.453 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6827", 2021-01-15T19:08:23.453 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.453 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.453 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.454 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:08:23.454 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.455 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6830", 2021-01-15T19:08:23.455 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.456 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6831", 2021-01-15T19:08:23.456 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.457 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.457 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:08:23.457 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:08:23.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:08:23.458 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6828", 2021-01-15T19:08:23.458 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:08:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6829", 2021-01-15T19:08:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:08:23.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.460 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.460 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6825/34874", 2021-01-15T19:08:23.460 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6827/34874", 2021-01-15T19:08:23.461 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6831/34874", 2021-01-15T19:08:23.461 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6829/34874", 2021-01-15T19:08:23.461 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:08:23.461 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:08:23.462 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:08:23.462 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.462 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:08:23.462 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2021-01-15T19:08:23.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.463 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:08:23.463 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.463 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.464 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.464 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.464 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.464 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.903933+0000", 2021-01-15T19:08:23.464 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.465 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:08:23.465 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.466 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.466 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.466 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.466 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.466 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.899879+0000", 2021-01-15T19:08:23.467 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.467 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:08:23.467 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.468 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.468 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.468 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.468 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.469 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.904316+0000", 2021-01-15T19:08:23.469 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.469 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:08:23.470 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.470 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.470 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.470 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.471 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.471 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.941511+0000", 2021-01-15T19:08:23.471 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.472 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:08:23.472 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.472 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.472 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.473 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.473 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.473 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.851510+0000", 2021-01-15T19:08:23.473 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.474 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:08:23.474 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.474 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.475 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.475 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.475 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.475 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.879200+0000", 2021-01-15T19:08:23.475 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.476 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:08:23.476 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.477 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.477 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.477 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.477 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.477 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.918755+0000", 2021-01-15T19:08:23.478 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.478 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:08:23.479 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:08:23.479 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:08:23.479 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:08:23.479 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:08:23.479 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:08:23.480 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:07:48.914415+0000", 2021-01-15T19:08:23.480 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:08:23.480 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.480 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:08:23.480 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2021-01-15T19:08:23.481 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2021-01-15T19:08:23.481 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2021-01-15T19:08:23.481 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2021-01-15T19:08:23.481 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2021-01-15T19:08:23.482 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2021-01-15T19:08:23.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2021-01-15T19:08:23.482 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:08:23.482 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:08:23.482 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:08:23.483 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2021-01-15T19:08:23.483 INFO:teuthology.orchestra.run.smithi001.stdout: "ruleset-failure-domain": "osd", 2021-01-15T19:08:23.483 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:08:23.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.484 INFO:teuthology.orchestra.run.smithi001.stdout: "isaprofile": { 2021-01-15T19:08:23.484 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-device-class": "", 2021-01-15T19:08:23.484 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:08:23.484 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-root": "default", 2021-01-15T19:08:23.484 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:08:23.485 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:08:23.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "isaprofile", 2021-01-15T19:08:23.485 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "isa", 2021-01-15T19:08:23.485 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:08:23.486 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.486 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [ 2021-01-15T19:08:23.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.486 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:08:23.487 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2021-01-15T19:08:23.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.487 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 2, 2021-01-15T19:08:23.487 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:08:23.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.488 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 6, 2021-01-15T19:08:23.488 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:08:23.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:23.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.489 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 9, 2021-01-15T19:08:23.489 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:08:23.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.489 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.490 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:08:23.490 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [ 2021-01-15T19:08:23.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.491 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:08:23.491 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2021-01-15T19:08:23.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:23.491 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 9, 2021-01-15T19:08:23.491 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:08:23.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:23.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.492 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:08:23.493 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2021-01-15T19:08:23.493 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2021-01-15T19:08:23.493 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2021-01-15T19:08:23.493 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2021-01-15T19:08:23.493 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2021-01-15T19:08:23.494 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2021-01-15T19:08:23.494 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2021-01-15T19:08:23.494 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2021-01-15T19:08:23.494 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2021-01-15T19:08:23.495 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:23.495 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:23.497 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:23.271+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:23.497 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:23.498 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:23.498 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:23.498 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:23.499 INFO:tasks.rados.rados.0.smithi001.stdout:90: done (3 left) 2021-01-15T19:08:23.499 INFO:tasks.rados.rados.0.smithi001.stdout:91: snap_create 2021-01-15T19:08:23.499 INFO:tasks.rados.rados.0.smithi001.stdout:87: finishing write tid 1 to smithi00140323-33 2021-01-15T19:08:23.499 INFO:tasks.rados.rados.0.smithi001.stdout:88: expect (ObjNum 43 snap 0 seq_num 43) 2021-01-15T19:08:23.500 INFO:tasks.rados.rados.0.smithi001.stdout:87: finishing write tid 2 to smithi00140323-33 2021-01-15T19:08:23.500 INFO:tasks.rados.rados.0.smithi001.stdout:87: finishing write tid 3 to smithi00140323-33 2021-01-15T19:08:23.500 INFO:tasks.rados.rados.0.smithi001.stdout:87: finishing write tid 4 to smithi00140323-33 2021-01-15T19:08:23.500 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 21 (ObjNum 58 snap 6 seq_num 58) dirty exists 2021-01-15T19:08:23.500 INFO:tasks.rados.rados.0.smithi001.stdout:87: left oid 33 (ObjNum 58 snap 6 seq_num 58) 2021-01-15T19:08:23.501 INFO:tasks.rados.rados.0.smithi001.stdout:89: finishing write tid 1 to smithi00140323-27 2021-01-15T19:08:23.501 INFO:tasks.rados.rados.0.smithi001.stdout:89: finishing write tid 2 to smithi00140323-27 2021-01-15T19:08:23.501 INFO:tasks.rados.rados.0.smithi001.stdout:89: finishing write tid 3 to smithi00140323-27 2021-01-15T19:08:23.501 INFO:tasks.rados.rados.0.smithi001.stdout:89: finishing write tid 4 to smithi00140323-27 2021-01-15T19:08:23.502 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 27 v 10 (ObjNum 59 snap 6 seq_num 59) dirty exists 2021-01-15T19:08:23.502 INFO:tasks.rados.rados.0.smithi001.stdout:89: left oid 27 (ObjNum 59 snap 6 seq_num 59) 2021-01-15T19:08:23.503 INFO:tasks.thrashosds.thrasher:j is {'epoch': 34, 'fsid': 'fe0e1890-043f-40bf-ad11-c5f08349f418', 'created': '2021-01-15T19:07:43.378003+0000', 'modified': '2021-01-15T19:08:22.263011+0000', 'last_up_change': '2021-01-15T19:07:50.512119+0000', 'last_in_change': '2021-01-15T19:07:50.512119+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 10, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e5b26f89-5c99-4764-9183-f3bb27c118f5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 34833}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 34833}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 34833}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 34833}]}, 'public_addr': '172.21.15.1:6801/34833', 'cluster_addr': '172.21.15.1:6803/34833', 'heartbeat_back_addr': '172.21.15.1:6807/34833', 'heartbeat_front_addr': '172.21.15.1:6805/34833', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '99c0d455-1bd6-43aa-bfec-b6bddf87a35e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 13, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 34834}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 34834}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 34834}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 34834}]}, 'public_addr': '172.21.15.1:6809/34834', 'cluster_addr': '172.21.15.1:6811/34834', 'heartbeat_back_addr': '172.21.15.1:6815/34834', 'heartbeat_front_addr': '172.21.15.1:6813/34834', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '71fe6c4b-3d73-4e07-b49d-98596ca7d9da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6817', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 34863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6821', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 34863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 34863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 34863}]}, 'public_addr': '172.21.15.1:6819/34863', 'cluster_addr': '172.21.15.1:6823/34863', 'heartbeat_back_addr': '172.21.15.1:6831/34863', 'heartbeat_front_addr': '172.21.15.1:6828/34863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8a6b018a-3a48-4a01-8526-a386f62ecb36', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6818', 'nonce': 34864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6822', 'nonce': 34864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 34864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 34864}]}, 'public_addr': '172.21.15.1:6818/34864', 'cluster_addr': '172.21.15.1:6822/34864', 'heartbeat_back_addr': '172.21.15.1:6829/34864', 'heartbeat_front_addr': '172.21.15.1:6825/34864', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ff089622-e642-4bd0-901c-60efdb87a549', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 34829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 34829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 34829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 34829}]}, 'public_addr': '172.21.15.136:6801/34829', 'cluster_addr': '172.21.15.136:6803/34829', 'heartbeat_back_addr': '172.21.15.136:6807/34829', 'heartbeat_front_addr': '172.21.15.136:6805/34829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3c57c228-dfaa-4e33-bded-27da0048d141', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 34830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 34830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6821', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 34830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6817', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 34830}]}, 'public_addr': '172.21.15.136:6811/34830', 'cluster_addr': '172.21.15.136:6815/34830', 'heartbeat_back_addr': '172.21.15.136:6823/34830', 'heartbeat_front_addr': '172.21.15.136:6819/34830', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '201f889f-dac1-4a26-a285-7a0ec21c3646', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6810', 'nonce': 34831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6814', 'nonce': 34831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6822', 'nonce': 34831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6818', 'nonce': 34831}]}, 'public_addr': '172.21.15.136:6810/34831', 'cluster_addr': '172.21.15.136:6814/34831', 'heartbeat_back_addr': '172.21.15.136:6822/34831', 'heartbeat_front_addr': '172.21.15.136:6818/34831', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '79ffd413-a9d3-4180-9691-d834d89c3f2e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 34874}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 34874}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 34874}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 34874}]}, 'public_addr': '172.21.15.136:6825/34874', 'cluster_addr': '172.21.15.136:6827/34874', 'heartbeat_back_addr': '172.21.15.136:6831/34874', 'heartbeat_front_addr': '172.21.15.136:6829/34874', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.903933+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.899879+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.904316+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.941511+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.851510+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.879200+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.918755+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:07:48.914415+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'isaprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '2', 'm': '1', 'name': 'isaprofile', 'plugin': 'isa', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}, {'begin': 6, 'length': 1}, {'begin': 9, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2021-01-15T19:08:23.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:23.504 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:23.494+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:23.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:23.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:23.529 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:23.550 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:23.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:23.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:23.592 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.594 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:23.597 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:23.596+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:23.663 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:23.664 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:23.673 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:23.699 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:23.697+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:23.800 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:23.801 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:23.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:23.801 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:23.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.801 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=62, reqid=client.4418.0:170, at_version=26'4, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:23.802 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:23.802 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5419753729999996, 2021-01-15T19:08:23.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999, 2021-01-15T19:08:23.802 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.803 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.803 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:23.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:23.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530000000000001e-05 2021-01-15T19:08:23.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256020+0000", 2021-01-15T19:08:23.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4555000000000001e-05 2021-01-15T19:08:23.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256044+0000", 2021-01-15T19:08:23.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9589000000000002e-05 2021-01-15T19:08:23.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256064+0000", 2021-01-15T19:08:23.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1316e-05 2021-01-15T19:08:23.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256075+0000", 2021-01-15T19:08:23.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3517999999999996e-05 2021-01-15T19:08:23.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256149+0000", 2021-01-15T19:08:23.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2123999999999999e-05 2021-01-15T19:08:23.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256161+0000", 2021-01-15T19:08:23.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025493310000000002 2021-01-15T19:08:23.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258710+0000", 2021-01-15T19:08:23.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0933999999999997e-05 2021-01-15T19:08:23.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258761+0000", 2021-01-15T19:08:23.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999 2021-01-15T19:08:23.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.815 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=64, reqid=client.4418.0:171, at_version=26'5, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:23.816 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:23.816 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5404147389999991, 2021-01-15T19:08:23.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632711, 2021-01-15T19:08:23.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.817 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.817 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:23.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257526+0000", 2021-01-15T19:08:23.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7207999999999999e-05 2021-01-15T19:08:23.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257573+0000", 2021-01-15T19:08:23.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2881e-05 2021-01-15T19:08:23.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257596+0000", 2021-01-15T19:08:23.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6541999999999998e-05 2021-01-15T19:08:23.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257613+0000", 2021-01-15T19:08:23.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4219999999999997e-06 2021-01-15T19:08:23.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257619+0000", 2021-01-15T19:08:23.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041151300000000002 2021-01-15T19:08:23.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258030+0000", 2021-01-15T19:08:23.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4518000000000001e-05 2021-01-15T19:08:23.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258045+0000", 2021-01-15T19:08:23.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001067995 2021-01-15T19:08:23.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259113+0000", 2021-01-15T19:08:23.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5631999999999999e-05 2021-01-15T19:08:23.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259159+0000", 2021-01-15T19:08:23.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001632711 2021-01-15T19:08:23.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.828 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.829 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=65, reqid=client.4418.0:172, at_version=26'6, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:23.829 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:23.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5387368220000006, 2021-01-15T19:08:23.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093191900000000002, 2021-01-15T19:08:23.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:23.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259204+0000", 2021-01-15T19:08:23.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9619e-05 2021-01-15T19:08:23.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259253+0000", 2021-01-15T19:08:23.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8291000000000002e-05 2021-01-15T19:08:23.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259272+0000", 2021-01-15T19:08:23.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6121e-05 2021-01-15T19:08:23.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259288+0000", 2021-01-15T19:08:23.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6279999999999999e-06 2021-01-15T19:08:23.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259294+0000", 2021-01-15T19:08:23.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4839000000000002e-05 2021-01-15T19:08:23.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259339+0000", 2021-01-15T19:08:23.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0299000000000001e-05 2021-01-15T19:08:23.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259350+0000", 2021-01-15T19:08:23.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073881299999999995 2021-01-15T19:08:23.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260088+0000", 2021-01-15T19:08:23.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7308999999999997e-05 2021-01-15T19:08:23.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260136+0000", 2021-01-15T19:08:23.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093191900000000002 2021-01-15T19:08:23.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.841 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.842 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=67, reqid=client.4418.0:172, at_version=0'0, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:23.842 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:23.842 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5372706879999996, 2021-01-15T19:08:23.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011872370000000001, 2021-01-15T19:08:23.843 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.843 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.843 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:23.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260670+0000", 2021-01-15T19:08:23.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6808000000000003e-05 2021-01-15T19:08:23.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260717+0000", 2021-01-15T19:08:23.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3665e-05 2021-01-15T19:08:23.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260730+0000", 2021-01-15T19:08:23.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5318000000000001e-05 2021-01-15T19:08:23.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260746+0000", 2021-01-15T19:08:23.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4309999999999999e-06 2021-01-15T19:08:23.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260752+0000", 2021-01-15T19:08:23.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5484e-05 2021-01-15T19:08:23.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260798+0000", 2021-01-15T19:08:23.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4531e-05 2021-01-15T19:08:23.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.260812+0000", 2021-01-15T19:08:23.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00098527100000000002 2021-01-15T19:08:23.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.261797+0000", 2021-01-15T19:08:23.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9728999999999997e-05 2021-01-15T19:08:23.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.261857+0000", 2021-01-15T19:08:23.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011872370000000001 2021-01-15T19:08:23.853 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.854 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.854 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:177 3.1s0 3:8f2f8304:::smithi00140323-26:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:23.855 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:23.855 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.516395631, 2021-01-15T19:08:23.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494, 2021-01-15T19:08:23.855 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.855 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:23.856 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:23.856 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:23.856 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:23.856 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2021-01-15T19:08:23.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.857 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:23.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:23.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6885000000000005e-05 2021-01-15T19:08:23.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281642+0000", 2021-01-15T19:08:23.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:08:23.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281650+0000", 2021-01-15T19:08:23.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2979999999999999e-05 2021-01-15T19:08:23.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281663+0000", 2021-01-15T19:08:23.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1930000000000001e-06 2021-01-15T19:08:23.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281669+0000", 2021-01-15T19:08:23.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880999999999999e-05 2021-01-15T19:08:23.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281724+0000", 2021-01-15T19:08:23.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946600000000001 2021-01-15T19:08:23.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:23.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281853+0000", 2021-01-15T19:08:23.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000582403 2021-01-15T19:08:23.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282435+0000", 2021-01-15T19:08:23.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013959510000000001 2021-01-15T19:08:23.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283831+0000", 2021-01-15T19:08:23.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010843630000000001 2021-01-15T19:08:23.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:23.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284916+0000", 2021-01-15T19:08:23.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5765000000000005e-05 2021-01-15T19:08:23.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284981+0000", 2021-01-15T19:08:23.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494 2021-01-15T19:08:23.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.869 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.869 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=27, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:23.870 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:23.870 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5140924029999994, 2021-01-15T19:08:23.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056741500000000004, 2021-01-15T19:08:23.870 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:23.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283848+0000", 2021-01-15T19:08:23.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5211999999999998e-05 2021-01-15T19:08:23.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283873+0000", 2021-01-15T19:08:23.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2769999999999996e-06 2021-01-15T19:08:23.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283879+0000", 2021-01-15T19:08:23.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0971999999999999e-05 2021-01-15T19:08:23.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283890+0000", 2021-01-15T19:08:23.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1859999999999997e-06 2021-01-15T19:08:23.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283896+0000", 2021-01-15T19:08:23.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050626700000000002 2021-01-15T19:08:23.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284402+0000", 2021-01-15T19:08:23.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3501e-05 2021-01-15T19:08:23.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284416+0000", 2021-01-15T19:08:23.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056741500000000004 2021-01-15T19:08:23.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.879 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.880 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 26/21 ECSubWriteReply(tid=27, last_complete=26'8, committed=1, applied=1))", 2021-01-15T19:08:23.880 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:23.881 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5133238510000009, 2021-01-15T19:08:23.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059160400000000002, 2021-01-15T19:08:23.881 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.881 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.881 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:23.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284617+0000", 2021-01-15T19:08:23.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7327999999999998e-05 2021-01-15T19:08:23.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284664+0000", 2021-01-15T19:08:23.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8720000000000008e-06 2021-01-15T19:08:23.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284673+0000", 2021-01-15T19:08:23.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6592e-05 2021-01-15T19:08:23.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284690+0000", 2021-01-15T19:08:23.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.725e-06 2021-01-15T19:08:23.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284696+0000", 2021-01-15T19:08:23.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7620000000000006e-05 2021-01-15T19:08:23.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284764+0000", 2021-01-15T19:08:23.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044446699999999998 2021-01-15T19:08:23.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.285208+0000", 2021-01-15T19:08:23.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059160400000000002 2021-01-15T19:08:23.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.890 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.890 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.891 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=71, reqid=client.4418.0:183, at_version=27'8, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:23.891 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:23.891 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5462438939999998, 2021-01-15T19:08:23.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003, 2021-01-15T19:08:23.891 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.892 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.892 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:23.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:23.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190392 2021-01-15T19:08:23.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251887+0000", 2021-01-15T19:08:23.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6526999999999999e-05 2021-01-15T19:08:23.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251914+0000", 2021-01-15T19:08:23.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8068999999999999e-05 2021-01-15T19:08:23.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251932+0000", 2021-01-15T19:08:23.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7269999999999996e-06 2021-01-15T19:08:23.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251940+0000", 2021-01-15T19:08:23.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012910800000000001 2021-01-15T19:08:23.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252070+0000", 2021-01-15T19:08:23.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2417e-05 2021-01-15T19:08:23.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252082+0000", 2021-01-15T19:08:23.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050434080000000001 2021-01-15T19:08:23.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257125+0000", 2021-01-15T19:08:23.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6032999999999998e-05 2021-01-15T19:08:23.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257171+0000", 2021-01-15T19:08:23.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003 2021-01-15T19:08:23.902 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.902 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.902 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.903 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=76, reqid=client.4418.0:184, at_version=27'9, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:23.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:23.904 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5434807080000006, 2021-01-15T19:08:23.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032503499999999999, 2021-01-15T19:08:23.904 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.904 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.904 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:23.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:23.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012968499999999999 2021-01-15T19:08:23.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254590+0000", 2021-01-15T19:08:23.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5418999999999999e-05 2021-01-15T19:08:23.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254605+0000", 2021-01-15T19:08:23.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999996e-06 2021-01-15T19:08:23.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254615+0000", 2021-01-15T19:08:23.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7790000000000003e-06 2021-01-15T19:08:23.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254619+0000", 2021-01-15T19:08:23.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3176000000000001e-05 2021-01-15T19:08:23.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254663+0000", 2021-01-15T19:08:23.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1525000000000001e-05 2021-01-15T19:08:23.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254674+0000", 2021-01-15T19:08:23.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030008919999999998 2021-01-15T19:08:23.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257675+0000", 2021-01-15T19:08:23.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5274000000000001e-05 2021-01-15T19:08:23.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257710+0000", 2021-01-15T19:08:23.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032503499999999999 2021-01-15T19:08:23.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.915 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:23.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:23.916 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5423925349999994, 2021-01-15T19:08:23.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:23.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:23.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:23.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:23.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:23.918 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:23.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.918 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:23.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:23.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:23.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:23.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:23.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:23.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:23.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:23.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:23.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:23.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:23.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:23.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:23.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:23.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:23.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:23.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:23.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:23.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:23.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:23.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:23.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:23.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:23.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:23.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:23.930 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.931 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.931 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.931 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=79, reqid=client.4418.0:185, at_version=27'10, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:23.932 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:23.932 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5411975760000001, 2021-01-15T19:08:23.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999, 2021-01-15T19:08:23.932 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.932 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.933 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:23.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:23.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5377000000000001e-05 2021-01-15T19:08:23.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256768+0000", 2021-01-15T19:08:23.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3387000000000001e-05 2021-01-15T19:08:23.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256782+0000", 2021-01-15T19:08:23.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6764999999999998e-05 2021-01-15T19:08:23.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256819+0000", 2021-01-15T19:08:23.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:23.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256826+0000", 2021-01-15T19:08:23.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9475999999999998e-05 2021-01-15T19:08:23.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256886+0000", 2021-01-15T19:08:23.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1929e-05 2021-01-15T19:08:23.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256898+0000", 2021-01-15T19:08:23.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055282589999999998 2021-01-15T19:08:23.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262426+0000", 2021-01-15T19:08:23.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2537999999999997e-05 2021-01-15T19:08:23.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262479+0000", 2021-01-15T19:08:23.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999 2021-01-15T19:08:23.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.943 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:23.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:23.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5396658950000006, 2021-01-15T19:08:23.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:23.945 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:23.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:23.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:23.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:23.946 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:23.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:23.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:23.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:23.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:23.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:23.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:23.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:23.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:23.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:23.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:23.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:23.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:23.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:23.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:23.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:23.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:23.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:23.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:23.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:23.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:23.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:23.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:23.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:23.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:23.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:23.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.959 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.959 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:23.960 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:23.960 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5387416859999998, 2021-01-15T19:08:23.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:23.960 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.960 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:23.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:23.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:23.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:23.961 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:23.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:23.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:23.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:23.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:23.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:23.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:23.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:23.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:23.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:23.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:23.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:23.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:23.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:23.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:23.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:23.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:23.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:23.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:23.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:23.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:23.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:23.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:23.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:23.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:23.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:23.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:23.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:23.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:23.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.538517723, 2021-01-15T19:08:23.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:23.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.976 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:23.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:23.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:23.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:23.977 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:23.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.978 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:23.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:23.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:23.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:23.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:23.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:23.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:23.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:23.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:23.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:23.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:23.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:23.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:23.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:23.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:23.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:23.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:23.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:23.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:23.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:23.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:23.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:23.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:23.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:23.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:23.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:23.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.990 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=81, reqid=client.4418.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=27'10))", 2021-01-15T19:08:23.990 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:23.990 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5349416159999993, 2021-01-15T19:08:23.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002, 2021-01-15T19:08:23.991 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:23.991 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:23.991 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:23.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:23.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:23.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:23.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4539999999999999e-05 2021-01-15T19:08:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:23.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263024+0000", 2021-01-15T19:08:23.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9600000000000006e-06 2021-01-15T19:08:23.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:23.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263032+0000", 2021-01-15T19:08:23.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2021-01-15T19:08:23.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:23.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263041+0000", 2021-01-15T19:08:23.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3239999999999996e-06 2021-01-15T19:08:23.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:23.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263046+0000", 2021-01-15T19:08:23.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4546999999999999e-05 2021-01-15T19:08:23.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:23.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263090+0000", 2021-01-15T19:08:23.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0596e-05 2021-01-15T19:08:23.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:23.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:23.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:23.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263101+0000", 2021-01-15T19:08:23.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052094810000000002 2021-01-15T19:08:23.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268310+0000", 2021-01-15T19:08:24.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4725000000000003e-05 2021-01-15T19:08:24.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268355+0000", 2021-01-15T19:08:24.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002 2021-01-15T19:08:24.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.002 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.003 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:24.003 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:24.003 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5346551379999998, 2021-01-15T19:08:24.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999, 2021-01-15T19:08:24.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:24.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:24.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3807000000000001e-05 2021-01-15T19:08:24.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263309+0000", 2021-01-15T19:08:24.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:24.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263317+0000", 2021-01-15T19:08:24.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2021-01-15T19:08:24.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263327+0000", 2021-01-15T19:08:24.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9009999999999997e-06 2021-01-15T19:08:24.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263334+0000", 2021-01-15T19:08:24.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052278899999999998 2021-01-15T19:08:24.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268562+0000", 2021-01-15T19:08:24.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4100000000000001e-05 2021-01-15T19:08:24.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268576+0000", 2021-01-15T19:08:24.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999 2021-01-15T19:08:24.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.012 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.013 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=30, last_complete=27'11, committed=1, applied=1))", 2021-01-15T19:08:24.013 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:24.013 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5339844290000002, 2021-01-15T19:08:24.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048998940000000001, 2021-01-15T19:08:24.014 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.014 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.014 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:24.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:24.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9382000000000001e-05 2021-01-15T19:08:24.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263986+0000", 2021-01-15T19:08:24.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.117e-06 2021-01-15T19:08:24.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263992+0000", 2021-01-15T19:08:24.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2400000000000007e-06 2021-01-15T19:08:24.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264000+0000", 2021-01-15T19:08:24.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2640000000000002e-06 2021-01-15T19:08:24.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264004+0000", 2021-01-15T19:08:24.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046074050000000002 2021-01-15T19:08:24.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268612+0000", 2021-01-15T19:08:24.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024448599999999998 2021-01-15T19:08:24.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268856+0000", 2021-01-15T19:08:24.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048998940000000001 2021-01-15T19:08:24.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.024 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:24.024 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.268190+0000", 2021-01-15T19:08:24.024 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5297502160000001, 2021-01-15T19:08:24.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066046200000000001, 2021-01-15T19:08:24.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.025 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.025 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268190+0000", 2021-01-15T19:08:24.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268190+0000", 2021-01-15T19:08:24.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3248999999999998e-05 2021-01-15T19:08:24.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268244+0000", 2021-01-15T19:08:24.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1276e-05 2021-01-15T19:08:24.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268255+0000", 2021-01-15T19:08:24.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6018000000000001e-05 2021-01-15T19:08:24.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268271+0000", 2021-01-15T19:08:24.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0809999999999994e-06 2021-01-15T19:08:24.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268279+0000", 2021-01-15T19:08:24.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000367828 2021-01-15T19:08:24.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268647+0000", 2021-01-15T19:08:24.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020400999999999999 2021-01-15T19:08:24.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268851+0000", 2021-01-15T19:08:24.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066046200000000001 2021-01-15T19:08:24.033 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.034 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=30, last_complete=27'11, committed=1, applied=1))", 2021-01-15T19:08:24.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.268327+0000", 2021-01-15T19:08:24.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5296135619999998, 2021-01-15T19:08:24.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072657099999999997, 2021-01-15T19:08:24.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.036 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268327+0000", 2021-01-15T19:08:24.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268327+0000", 2021-01-15T19:08:24.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9302999999999999e-05 2021-01-15T19:08:24.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268376+0000", 2021-01-15T19:08:24.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2259999999999999e-06 2021-01-15T19:08:24.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268384+0000", 2021-01-15T19:08:24.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2412000000000001e-05 2021-01-15T19:08:24.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268396+0000", 2021-01-15T19:08:24.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1930000000000003e-06 2021-01-15T19:08:24.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268400+0000", 2021-01-15T19:08:24.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047790799999999998 2021-01-15T19:08:24.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268878+0000", 2021-01-15T19:08:24.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017552900000000001 2021-01-15T19:08:24.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269054+0000", 2021-01-15T19:08:24.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072657099999999997 2021-01-15T19:08:24.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.044 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.045 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=31, last_complete=27'12, committed=1, applied=1))", 2021-01-15T19:08:24.045 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.270526+0000", 2021-01-15T19:08:24.045 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5274148790000002, 2021-01-15T19:08:24.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000478746, 2021-01-15T19:08:24.045 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.046 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.046 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270526+0000", 2021-01-15T19:08:24.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270526+0000", 2021-01-15T19:08:24.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0468e-05 2021-01-15T19:08:24.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270556+0000", 2021-01-15T19:08:24.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0440000000000003e-06 2021-01-15T19:08:24.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270561+0000", 2021-01-15T19:08:24.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2199999999999998e-06 2021-01-15T19:08:24.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270570+0000", 2021-01-15T19:08:24.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0420000000000002e-06 2021-01-15T19:08:24.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270576+0000", 2021-01-15T19:08:24.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4993000000000002e-05 2021-01-15T19:08:24.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270611+0000", 2021-01-15T19:08:24.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039397899999999998 2021-01-15T19:08:24.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271004+0000", 2021-01-15T19:08:24.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000478746 2021-01-15T19:08:24.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.054 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.055 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.055 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:24.055 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:23.798+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:24.058 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:24.058 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2021-01-15T19:08:23.510770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27516928,"num_objects":58,"num_object_clones":7,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":1,"num_read":146,"num_read_kb":10506,"num_write":170,"num_write_kb":27057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168,"ondisk_log_size":168,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84308,"kb_used_data":42964,"kb_used_omap":0,"kb_used_meta":41280,"kb_avail":754890412,"statfs":{"total":773094113280,"available":773007781888,"internally_reserved":0,"allocated":43995136,"data_stored":45718800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42270720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1482752,"num_objects":-2,"num_object_clones":-1,"num_object_copies":-6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":-1,"num_read":14,"num_read_kb":1588,"num_write":24,"num_write_kb":1600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000729"},"pg_stats":[{"pgid":"3.f","version":"32'16","reported_seq":"47","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:21.155871+0000","last_change":"2021-01-15T19:08:21.156273+0000","last_active":"2021-01-15T19:08:21.155871+0000","last_peered":"2021-01-15T19:08:21.155871+0000","last_clean":"2021-01-15T19:08:21.155871+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:21.155871+0000","last_undegraded":"2021-01-15T19:08:21.155871+0000","last_fullsized":"2021-01-15T19:08:21.155871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2646016,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":893,"num_write":10,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.e","version":"25'4","reported_seq":"36","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.260434+0000","last_change":"2021-01-15T19:08:20.260749+0000","last_active":"2021-01-15T19:08:20.260434+0000","last_peered":"2021-01-15T19:08:20.260434+0000","last_clean":"2021-01-15T19:08:20.260434+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:20.260434+0000","last_undegraded":"2021-01-15T19:08:20.260434+0000","last_fullsized":"2021-01-15T19:08:20.260434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":555,"num_write":4,"num_write_kb":848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.d","version":"29'8","reported_seq":"35","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.261574+0000","last_change":"2021-01-15T19:08:20.261916+0000","last_active":"2021-01-15T19:08:20.261574+0000","last_peered":"2021-01-15T19:08:20.261574+0000","last_clean":"2021-01-15T19:08:20.261574+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:20.261574+0000","last_undegraded":"2021-01-15T19:08:20.261574+0000","last_fullsized":"2021-01-15T19:08:20.261574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":417792,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":26,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.c","version":"28'18","reported_seq":"50","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.653740+0000","last_change":"2021-01-15T19:08:20.654059+0000","last_active":"2021-01-15T19:08:20.653740+0000","last_peered":"2021-01-15T19:08:20.653740+0000","last_clean":"2021-01-15T19:08:20.653740+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:20.653740+0000","last_undegraded":"2021-01-15T19:08:20.653740+0000","last_fullsized":"2021-01-15T19:08:20.653740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2965504,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":16,"num_read_kb":1639,"num_write":14,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.b","version":"25'4","reported_seq":"34","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.260071+0000","last_change":"2021-01-15T19:08:20.260071+0000","last_active":"2021-01-15T19:08:20.260071+0000","last_peered":"2021-01-15T19:08:20.260071+0000","last_clean":"2021-01-15T19:08:20.260071+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:20.260071+0000","last_undegraded":"2021-01-15T19:08:20.260071+0000","last_fullsized":"2021-01-15T19:08:20.260071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1220608,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":1192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.a","version":"25'6","reported_seq":"35","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.261392+0000","last_change":"2021-01-15T19:08:20.261704+0000","last_active":"2021-01-15T19:08:20.261392+0000","last_peered":"2021-01-15T19:08:20.261392+0000","last_clean":"2021-01-15T19:08:20.261392+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:20.261392+0000","last_undegraded":"2021-01-15T19:08:20.261392+0000","last_fullsized":"2021-01-15T19:08:20.261392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1843200,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.9","version":"28'17","reported_seq":"50","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.260318+0000","last_change":"2021-01-15T19:08:20.260570+0000","last_active":"2021-01-15T19:08:20.260318+0000","last_peered":"2021-01-15T19:08:20.260318+0000","last_clean":"2021-01-15T19:08:20.260318+0000","last_became_active":"2021-01-15T19:08:10.230943+0000","last_became_peered":"2021-01-15T19:08:10.230943+0000","last_unstale":"2021-01-15T19:08:20.260318+0000","last_undegraded":"2021-01-15T19:08:20.260318+0000","last_fullsized":"2021-01-15T19:08:20.260318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3129344,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":1831,"num_write":14,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"3.8","version":"28'16","reported_seq":"43","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.264867+0000","last_change":"2021-01-15T19:08:16.759883+0000","last_active":"2021-01-15T19:08:18.264867+0000","last_peered":"2021-01-15T19:08:18.264867+0000","last_clean":"2021-01-15T19:08:18.264867+0000","last_became_active":"2021-01-15T19:08:10.231245+0000","last_became_peered":"2021-01-15T19:08:10.231245+0000","last_unstale":"2021-01-15T19:08:18.264867+0000","last_undegraded":"2021-01-15T19:08:18.264867+0000","last_fullsized":"2021-01-15T19:08:18.264867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2547712,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":12,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"20","reported_epoch":"29","state":"active+clean","last_fresh":"2021-01-15T19:08:17.255516+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:08:17.255516+0000","last_peered":"2021-01-15T19:08:17.255516+0000","last_clean":"2021-01-15T19:08:17.255516+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:08:17.255516+0000","last_undegraded":"2021-01-15T19:08:17.255516+0000","last_fullsized":"2021-01-15T19:08:17.255516+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"35","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.259815+0000","last_change":"2021-01-15T19:08:20.260246+0000","last_active":"2021-01-15T19:08:20.259815+0000","last_peered":"2021-01-15T19:08:20.259815+0000","last_clean":"2021-01-15T19:08:20.259815+0000","last_became_active":"2021-01-15T19:08:10.230610+0000","last_became_peered":"2021-01-15T19:08:10.230610+0000","last_unstale":"2021-01-15T19:08:20.259815+0000","last_undegraded":"2021-01-15T19:08:20.259815+0000","last_fullsized":"2021-01-15T19:08:20.259815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1548288,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"20","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.252877+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:08:18.252877+0000","last_peered":"2021-01-15T19:08:18.252877+0000","last_clean":"2021-01-15T19:08:18.252877+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:08:18.252877+0000","last_undegraded":"2021-01-15T19:08:18.252877+0000","last_fullsized":"2021-01-15T19:08:18.252877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"25","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.653461+0000","last_change":"2021-01-15T19:08:20.653677+0000","last_active":"2021-01-15T19:08:20.653461+0000","last_peered":"2021-01-15T19:08:20.653461+0000","last_clean":"2021-01-15T19:08:20.653461+0000","last_became_active":"2021-01-15T19:08:10.237812+0000","last_became_peered":"2021-01-15T19:08:10.237812+0000","last_unstale":"2021-01-15T19:08:20.653461+0000","last_undegraded":"2021-01-15T19:08:20.653461+0000","last_fullsized":"2021-01-15T19:08:20.653461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"24","reported_epoch":"32","state":"active+clean","last_fresh":"2021-01-15T19:08:20.259686+0000","last_change":"2021-01-15T19:07:56.182164+0000","last_active":"2021-01-15T19:08:20.259686+0000","last_peered":"2021-01-15T19:08:20.259686+0000","last_clean":"2021-01-15T19:08:20.259686+0000","last_became_active":"2021-01-15T19:07:56.181662+0000","last_became_peered":"2021-01-15T19:07:56.181662+0000","last_unstale":"2021-01-15T19:08:20.259686+0000","last_undegraded":"2021-01-15T19:08:20.259686+0000","last_fullsized":"2021-01-15T19:08:20.259686+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'10","reported_seq":"45","reported_epoch":"29","state":"active+clean","last_fresh":"2021-01-15T19:08:17.259920+0000","last_change":"2021-01-15T19:08:16.252960+0000","last_active":"2021-01-15T19:08:17.259920+0000","last_peered":"2021-01-15T19:08:17.259920+0000","last_clean":"2021-01-15T19:08:17.259920+0000","last_became_active":"2021-01-15T19:08:10.231211+0000","last_became_peered":"2021-01-15T19:08:10.231211+0000","last_unstale":"2021-01-15T19:08:17.259920+0000","last_undegraded":"2021-01-15T19:08:17.259920+0000","last_fullsized":"2021-01-15T19:08:17.259920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1523712,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":1928,"num_write":10,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":"20","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.252897+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:08:18.252897+0000","last_peered":"2021-01-15T19:08:18.252897+0000","last_clean":"2021-01-15T19:08:18.252897+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:08:18.252897+0000","last_undegraded":"2021-01-15T19:08:18.252897+0000","last_fullsized":"2021-01-15T19:08:18.252897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"27","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.262572+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:18.262572+0000","last_peered":"2021-01-15T19:08:18.262572+0000","last_clean":"2021-01-15T19:08:18.262572+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:18.262572+0000","last_undegraded":"2021-01-15T19:08:18.262572+0000","last_fullsized":"2021-01-15T19:08:18.262572+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"28'12","reported_seq":"40","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.264913+0000","last_change":"2021-01-15T19:08:16.765034+0000","last_active":"2021-01-15T19:08:18.264913+0000","last_peered":"2021-01-15T19:08:18.264913+0000","last_clean":"2021-01-15T19:08:18.264913+0000","last_became_active":"2021-01-15T19:08:10.234192+0000","last_became_peered":"2021-01-15T19:08:10.234192+0000","last_unstale":"2021-01-15T19:08:18.264913+0000","last_undegraded":"2021-01-15T19:08:18.264913+0000","last_fullsized":"2021-01-15T19:08:18.264913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1974272,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3221,"num_write":9,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"20","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.252838+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:08:18.252838+0000","last_peered":"2021-01-15T19:08:18.252838+0000","last_clean":"2021-01-15T19:08:18.252838+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:08:18.252838+0000","last_undegraded":"2021-01-15T19:08:18.252838+0000","last_fullsized":"2021-01-15T19:08:18.252838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"28'16","reported_seq":"38","reported_epoch":"29","state":"active+clean","last_fresh":"2021-01-15T19:08:17.261147+0000","last_change":"2021-01-15T19:08:16.944117+0000","last_active":"2021-01-15T19:08:17.261147+0000","last_peered":"2021-01-15T19:08:17.261147+0000","last_clean":"2021-01-15T19:08:17.261147+0000","last_became_active":"2021-01-15T19:08:10.230297+0000","last_became_peered":"2021-01-15T19:08:10.230297+0000","last_unstale":"2021-01-15T19:08:17.261147+0000","last_undegraded":"2021-01-15T19:08:17.261147+0000","last_fullsized":"2021-01-15T19:08:17.261147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1966080,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":29,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"20","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.252996+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:08:18.252996+0000","last_peered":"2021-01-15T19:08:18.252996+0000","last_clean":"2021-01-15T19:08:18.252996+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:08:18.252996+0000","last_undegraded":"2021-01-15T19:08:18.252996+0000","last_fullsized":"2021-01-15T19:08:18.252996+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'6","reported_seq":"30","reported_epoch":"29","state":"active+clean","last_fresh":"2021-01-15T19:08:17.261278+0000","last_change":"2021-01-15T19:08:16.435438+0000","last_active":"2021-01-15T19:08:17.261278+0000","last_peered":"2021-01-15T19:08:17.261278+0000","last_clean":"2021-01-15T19:08:17.261278+0000","last_became_active":"2021-01-15T19:08:10.228436+0000","last_became_peered":"2021-01-15T19:08:10.228436+0000","last_unstale":"2021-01-15T19:08:17.261278+0000","last_undegraded":"2021-01-15T19:08:17.261278+0000","last_fullsized":"2021-01-15T19:08:17.261278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1163264,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.7","version":"29'14","reported_seq":"36","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.265971+0000","last_change":"2021-01-15T19:08:17.267658+0000","last_active":"2021-01-15T19:08:18.265971+0000","last_peered":"2021-01-15T19:08:18.265971+0000","last_clean":"2021-01-15T19:08:18.265971+0000","last_became_active":"2021-01-15T19:08:10.703898+0000","last_became_peered":"2021-01-15T19:08:10.703898+0000","last_unstale":"2021-01-15T19:08:18.265971+0000","last_undegraded":"2021-01-15T19:08:18.265971+0000","last_fullsized":"2021-01-15T19:08:18.265971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2228224,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.265117+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:08:18.265117+0000","last_peered":"2021-01-15T19:08:18.265117+0000","last_clean":"2021-01-15T19:08:18.265117+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:08:18.265117+0000","last_undegraded":"2021-01-15T19:08:18.265117+0000","last_fullsized":"2021-01-15T19:08:18.265117+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"20","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.252918+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:08:18.252918+0000","last_peered":"2021-01-15T19:08:18.252918+0000","last_clean":"2021-01-15T19:08:18.252918+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:08:18.252918+0000","last_undegraded":"2021-01-15T19:08:18.252918+0000","last_fullsized":"2021-01-15T19:08:18.252918+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"28'13","reported_seq":"41","reported_epoch":"30","state":"active+clean","last_fresh":"2021-01-15T19:08:18.265008+0000","last_change":"2021-01-15T19:08:17.262992+0000","last_active":"2021-01-15T19:08:18.265008+0000","last_peered":"2021-01-15T19:08:18.265008+0000","last_clean":"2021-01-15T19:08:18.265008+0000","last_became_active":"2021-01-15T19:08:10.703953+0000","last_became_peered":"2021-01-15T19:08:10.703953+0000","last_unstale":"2021-01-15T19:08:18.265008+0000","last_undegraded":"2021-01-15T19:08:18.265008+0000","last_fullsized":"2021-01-15T19:08:18.265008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1474560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":413,"num_write":10,"num_write_kb":1592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":27516928,"num_objects":56,"num_object_clones":7,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":1,"num_read":146,"num_read_kb":10506,"num_write":168,"num_write_kb":27048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":41275392,"data_stored":44568576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11124,"kb_used_data":5996,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94360716,"statfs":{"total":96636764160,"available":96625373184,"internally_reserved":0,"allocated":6139904,"data_stored":6294343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9480,"kb_used_data":4288,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94362360,"statfs":{"total":96636764160,"available":96627056640,"internally_reserved":0,"allocated":4390912,"data_stored":4630561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9664,"kb_used_data":4536,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":94362176,"statfs":{"total":96636764160,"available":96626868224,"internally_reserved":0,"allocated":4644864,"data_stored":5079440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.165}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1280000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10108,"kb_used_data":5044,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94361732,"statfs":{"total":96636764160,"available":96626413568,"internally_reserved":0,"allocated":5165056,"data_stored":5126177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12208,"kb_used_data":6952,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94359632,"statfs":{"total":96636764160,"available":96624263168,"internally_reserved":0,"allocated":7118848,"data_stored":7389584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9488,"kb_used_data":4424,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94362352,"statfs":{"total":96636764160,"available":96627048448,"internally_reserved":0,"allocated":4530176,"data_stored":4488007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.617}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15168,"kb_used_data":9656,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":94356672,"statfs":{"total":96636764160,"available":96621232128,"internally_reserved":0,"allocated":9887744,"data_stored":10789264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7068,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94364772,"statfs":{"total":96636764160,"available":96629526528,"internally_reserved":0,"allocated":2117632,"data_stored":1921424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]}]}} 2021-01-15T19:08:24.059 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:24.061 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [5, 2, 4, 0] 2021-01-15T19:08:24.061 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.1', '5', '2', '4', '0'] 2021-01-15T19:08:24.061 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.1 5 2 4 0 2021-01-15T19:08:24.062 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:24.055+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:24.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:24.157 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:24.156+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:24.226 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:24.227 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:24.227 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:24.227 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:24.237 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:24.257 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:24.256+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:24.277 INFO:tasks.rados.rados.0.smithi001.stdout:87: done (3 left) 2021-01-15T19:08:24.277 INFO:tasks.rados.rados.0.smithi001.stdout:88: done (2 left) 2021-01-15T19:08:24.278 INFO:tasks.rados.rados.0.smithi001.stdout:89: done (1 left) 2021-01-15T19:08:24.278 INFO:tasks.rados.rados.0.smithi001.stdout:91: done (0 left) 2021-01-15T19:08:24.279 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 40 current snap is 7 2021-01-15T19:08:24.279 INFO:tasks.rados.rados.0.smithi001.stdout:92: seq_num 60 ranges {663552=614400,1277952=401408} 2021-01-15T19:08:24.284 INFO:tasks.rados.rados.0.smithi001.stdout:92: writing smithi00140323-40 from 663552 to 1277952 tid 1 2021-01-15T19:08:24.286 INFO:tasks.rados.rados.0.smithi001.stdout:92: writing smithi00140323-40 from 1277952 to 1679360 tid 2 2021-01-15T19:08:24.287 INFO:tasks.rados.rados.0.smithi001.stdout:93: delete oid 47 current snap is 7 2021-01-15T19:08:24.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:08:24.297 INFO:tasks.rados.rados.0.smithi001.stdout:92: finishing write tid 1 to smithi00140323-40 2021-01-15T19:08:24.299 INFO:tasks.rados.rados.0.smithi001.stdout:92: finishing write tid 2 to smithi00140323-40 2021-01-15T19:08:24.299 INFO:tasks.rados.rados.0.smithi001.stdout:93: done (1 left) 2021-01-15T19:08:24.299 INFO:tasks.rados.rados.0.smithi001.stdout:94: rmattr oid 30 current snap is 7 2021-01-15T19:08:24.300 INFO:tasks.rados.rados.0.smithi001.stdout:94: done (1 left) 2021-01-15T19:08:24.300 INFO:tasks.rados.rados.0.smithi001.stdout:95: setattr oid 19 current snap is 7 2021-01-15T19:08:24.300 INFO:tasks.rados.rados.0.smithi001.stdout:92: finishing write tid 3 to smithi00140323-40 2021-01-15T19:08:24.301 INFO:tasks.rados.rados.0.smithi001.stdout:96: read oid 36 snap -1 2021-01-15T19:08:24.301 INFO:tasks.rados.rados.0.smithi001.stdout:96: expect (ObjNum 35 snap 0 seq_num 35) 2021-01-15T19:08:24.302 INFO:tasks.rados.rados.0.smithi001.stdout:97: read oid 26 snap 4 2021-01-15T19:08:24.302 INFO:tasks.rados.rados.0.smithi001.stdout:97: expect (ObjNum 25 snap 0 seq_num 25) 2021-01-15T19:08:24.302 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 34 current snap is 7 2021-01-15T19:08:24.302 INFO:tasks.rados.rados.0.smithi001.stdout:98: seq_num 62 ranges {557056=647168,1204224=401408} 2021-01-15T19:08:24.308 INFO:tasks.rados.rados.0.smithi001.stdout:98: writing smithi00140323-34 from 557056 to 1204224 tid 1 2021-01-15T19:08:24.311 INFO:tasks.rados.rados.0.smithi001.stdout:98: writing smithi00140323-34 from 1204224 to 1605632 tid 2 2021-01-15T19:08:24.311 INFO:tasks.rados.rados.0.smithi001.stdout:92: finishing write tid 4 to smithi00140323-40 2021-01-15T19:08:24.311 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 40 v 16 (ObjNum 60 snap 7 seq_num 60) dirty exists 2021-01-15T19:08:24.312 INFO:tasks.rados.rados.0.smithi001.stdout:92: left oid 40 (ObjNum 60 snap 7 seq_num 60) 2021-01-15T19:08:24.312 INFO:tasks.rados.rados.0.smithi001.stdout:97: expect (ObjNum 25 snap 0 seq_num 25) 2021-01-15T19:08:24.328 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 20 (ObjNum 57 snap 4 seq_num 57) dirty exists 2021-01-15T19:08:24.328 INFO:tasks.rados.rados.0.smithi001.stdout:92: done (4 left) 2021-01-15T19:08:24.329 INFO:tasks.rados.rados.0.smithi001.stdout:95: done (3 left) 2021-01-15T19:08:24.329 INFO:tasks.rados.rados.0.smithi001.stdout:97: done (2 left) 2021-01-15T19:08:24.329 INFO:tasks.rados.rados.0.smithi001.stdout:99: snap_remove snap 6 2021-01-15T19:08:24.358 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:24.357+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:24.409 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:24.410 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:24.419 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:24.459 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:24.457+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:24.560 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:24.558+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:24.585 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:24.585 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:24.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:24.586 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:24.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.586 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=34, reqid=client.4418.0:156, at_version=26'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:24.586 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:24.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.339633667999999, 2021-01-15T19:08:24.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002, 2021-01-15T19:08:24.587 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.587 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.587 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:24.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:24.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166446 2021-01-15T19:08:24.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243528+0000", 2021-01-15T19:08:24.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5724e-05 2021-01-15T19:08:24.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243554+0000", 2021-01-15T19:08:24.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3528e-05 2021-01-15T19:08:24.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243567+0000", 2021-01-15T19:08:24.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4610000000000008e-06 2021-01-15T19:08:24.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243576+0000", 2021-01-15T19:08:24.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1644999999999999e-05 2021-01-15T19:08:24.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243618+0000", 2021-01-15T19:08:24.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2930000000000001e-06 2021-01-15T19:08:24.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243626+0000", 2021-01-15T19:08:24.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020585310000000002 2021-01-15T19:08:24.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245684+0000", 2021-01-15T19:08:24.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7432999999999997e-05 2021-01-15T19:08:24.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245742+0000", 2021-01-15T19:08:24.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002 2021-01-15T19:08:24.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=36, reqid=client.4418.0:157, at_version=26'13, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:24.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:24.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.337151167, 2021-01-15T19:08:24.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018949259999999999, 2021-01-15T19:08:24.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.600 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:24.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245844+0000", 2021-01-15T19:08:24.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019949400000000001 2021-01-15T19:08:24.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246044+0000", 2021-01-15T19:08:24.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3130999999999999e-05 2021-01-15T19:08:24.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246067+0000", 2021-01-15T19:08:24.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8967999999999999e-05 2021-01-15T19:08:24.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246086+0000", 2021-01-15T19:08:24.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7729999999999999e-06 2021-01-15T19:08:24.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246094+0000", 2021-01-15T19:08:24.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3356999999999998e-05 2021-01-15T19:08:24.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246137+0000", 2021-01-15T19:08:24.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0242999999999999e-05 2021-01-15T19:08:24.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.246157+0000", 2021-01-15T19:08:24.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015296000000000001 2021-01-15T19:08:24.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.247687+0000", 2021-01-15T19:08:24.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.236e-05 2021-01-15T19:08:24.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.247739+0000", 2021-01-15T19:08:24.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018949259999999999 2021-01-15T19:08:24.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.616 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=39, reqid=client.4418.0:158, at_version=0'0, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:24.617 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:24.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.325081598000001, 2021-01-15T19:08:24.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001101858, 2021-01-15T19:08:24.617 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.618 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:24.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257914+0000", 2021-01-15T19:08:24.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5347999999999999e-05 2021-01-15T19:08:24.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257969+0000", 2021-01-15T19:08:24.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5548000000000001e-05 2021-01-15T19:08:24.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.257985+0000", 2021-01-15T19:08:24.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8144000000000001e-05 2021-01-15T19:08:24.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258003+0000", 2021-01-15T19:08:24.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2830000000000004e-06 2021-01-15T19:08:24.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258010+0000", 2021-01-15T19:08:24.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135764 2021-01-15T19:08:24.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258146+0000", 2021-01-15T19:08:24.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3592e-05 2021-01-15T19:08:24.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258159+0000", 2021-01-15T19:08:24.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080877599999999998 2021-01-15T19:08:24.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258968+0000", 2021-01-15T19:08:24.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7403e-05 2021-01-15T19:08:24.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.259016+0000", 2021-01-15T19:08:24.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001101858 2021-01-15T19:08:24.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.629 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.630 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 26/21 ECSubWrite(tid=69, reqid=client.4418.0:173, at_version=26'11, trim_to=0'0, roll_forward_to=25'9))", 2021-01-15T19:08:24.630 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:24.630 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.304384169, 2021-01-15T19:08:24.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087352800000000004, 2021-01-15T19:08:24.631 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.631 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.631 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:24.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278611+0000", 2021-01-15T19:08:24.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6239000000000002e-05 2021-01-15T19:08:24.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278657+0000", 2021-01-15T19:08:24.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7796e-05 2021-01-15T19:08:24.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278685+0000", 2021-01-15T19:08:24.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8410000000000002e-05 2021-01-15T19:08:24.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278704+0000", 2021-01-15T19:08:24.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0550999999999999e-05 2021-01-15T19:08:24.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278714+0000", 2021-01-15T19:08:24.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0232999999999999e-05 2021-01-15T19:08:24.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278754+0000", 2021-01-15T19:08:24.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0690000000000008e-06 2021-01-15T19:08:24.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.278764+0000", 2021-01-15T19:08:24.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067577100000000003 2021-01-15T19:08:24.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.279439+0000", 2021-01-15T19:08:24.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5459e-05 2021-01-15T19:08:24.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.279485+0000", 2021-01-15T19:08:24.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087352800000000004 2021-01-15T19:08:24.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.643 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.643 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=21, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:24.644 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:24.644 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.300634196000001, 2021-01-15T19:08:24.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010378760000000001, 2021-01-15T19:08:24.644 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.644 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:24.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282361+0000", 2021-01-15T19:08:24.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4665e-05 2021-01-15T19:08:24.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282406+0000", 2021-01-15T19:08:24.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2330999999999999e-05 2021-01-15T19:08:24.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282418+0000", 2021-01-15T19:08:24.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8114e-05 2021-01-15T19:08:24.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282436+0000", 2021-01-15T19:08:24.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4099999999999997e-06 2021-01-15T19:08:24.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282446+0000", 2021-01-15T19:08:24.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120245 2021-01-15T19:08:24.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282566+0000", 2021-01-15T19:08:24.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083311099999999999 2021-01-15T19:08:24.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283399+0000", 2021-01-15T19:08:24.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010378760000000001 2021-01-15T19:08:24.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.653 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.654 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s1 26/21 ECSubRead(tid=22, to_read={3:0c50693c:::smithi00140323-16:head=0,126976,0}, subchunks={3:0c50693c:::smithi00140323-16:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:24.654 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:24.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.300281595, 2021-01-15T19:08:24.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090976900000000005, 2021-01-15T19:08:24.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.655 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.655 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:24.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282714+0000", 2021-01-15T19:08:24.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9024999999999999e-05 2021-01-15T19:08:24.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282763+0000", 2021-01-15T19:08:24.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2038999999999999e-05 2021-01-15T19:08:24.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282775+0000", 2021-01-15T19:08:24.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9735999999999999e-05 2021-01-15T19:08:24.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282795+0000", 2021-01-15T19:08:24.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4930000000000003e-06 2021-01-15T19:08:24.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282801+0000", 2021-01-15T19:08:24.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063796299999999996 2021-01-15T19:08:24.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283439+0000", 2021-01-15T19:08:24.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184513 2021-01-15T19:08:24.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283624+0000", 2021-01-15T19:08:24.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090976900000000005 2021-01-15T19:08:24.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=74, reqid=client.4418.0:190, at_version=27'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:24.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:24.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3311625280000001, 2021-01-15T19:08:24.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002, 2021-01-15T19:08:24.666 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.666 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:24.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:24.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8488000000000001e-05 2021-01-15T19:08:24.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251871+0000", 2021-01-15T19:08:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0378e-05 2021-01-15T19:08:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251902+0000", 2021-01-15T19:08:24.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3121e-05 2021-01-15T19:08:24.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251915+0000", 2021-01-15T19:08:24.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3499999999999996e-06 2021-01-15T19:08:24.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251920+0000", 2021-01-15T19:08:24.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108286 2021-01-15T19:08:24.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252028+0000", 2021-01-15T19:08:24.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1652e-05 2021-01-15T19:08:24.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252040+0000", 2021-01-15T19:08:24.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002924991 2021-01-15T19:08:24.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254965+0000", 2021-01-15T19:08:24.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4570000000000001e-05 2021-01-15T19:08:24.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255020+0000", 2021-01-15T19:08:24.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002 2021-01-15T19:08:24.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.676 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.677 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=75, reqid=client.4418.0:191, at_version=27'13, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:24.677 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:24.678 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.329655399, 2021-01-15T19:08:24.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022390299999999999, 2021-01-15T19:08:24.678 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.678 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:24.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:24.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0649999999999998e-05 2021-01-15T19:08:24.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253391+0000", 2021-01-15T19:08:24.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0658000000000001e-05 2021-01-15T19:08:24.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253411+0000", 2021-01-15T19:08:24.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8773000000000001e-05 2021-01-15T19:08:24.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253430+0000", 2021-01-15T19:08:24.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4130000000000006e-06 2021-01-15T19:08:24.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253438+0000", 2021-01-15T19:08:24.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4926000000000003e-05 2021-01-15T19:08:24.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253483+0000", 2021-01-15T19:08:24.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3898e-05 2021-01-15T19:08:24.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253497+0000", 2021-01-15T19:08:24.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020547130000000001 2021-01-15T19:08:24.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255552+0000", 2021-01-15T19:08:24.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6999e-05 2021-01-15T19:08:24.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255579+0000", 2021-01-15T19:08:24.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022390299999999999 2021-01-15T19:08:24.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.688 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=78, reqid=client.4418.0:192, at_version=27'14, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:24.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256226+0000", 2021-01-15T19:08:24.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.326769702, 2021-01-15T19:08:24.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001558516, 2021-01-15T19:08:24.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.690 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256226+0000", 2021-01-15T19:08:24.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256226+0000", 2021-01-15T19:08:24.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0145999999999997e-05 2021-01-15T19:08:24.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256276+0000", 2021-01-15T19:08:24.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3802e-05 2021-01-15T19:08:24.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256300+0000", 2021-01-15T19:08:24.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8654000000000001e-05 2021-01-15T19:08:24.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256318+0000", 2021-01-15T19:08:24.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7539999999999995e-06 2021-01-15T19:08:24.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256327+0000", 2021-01-15T19:08:24.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2339999999999997e-05 2021-01-15T19:08:24.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256379+0000", 2021-01-15T19:08:24.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1408e-05 2021-01-15T19:08:24.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256391+0000", 2021-01-15T19:08:24.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001337792 2021-01-15T19:08:24.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257729+0000", 2021-01-15T19:08:24.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5619999999999999e-05 2021-01-15T19:08:24.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257784+0000", 2021-01-15T19:08:24.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001558516 2021-01-15T19:08:24.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.701 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=80, reqid=client.4418.0:192, at_version=0'0, trim_to=0'0, roll_forward_to=27'14))", 2021-01-15T19:08:24.701 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258427+0000", 2021-01-15T19:08:24.701 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3245688619999996, 2021-01-15T19:08:24.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088728699999999995, 2021-01-15T19:08:24.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258427+0000", 2021-01-15T19:08:24.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258427+0000", 2021-01-15T19:08:24.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6684999999999999e-05 2021-01-15T19:08:24.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258453+0000", 2021-01-15T19:08:24.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0017000000000001e-05 2021-01-15T19:08:24.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258463+0000", 2021-01-15T19:08:24.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0667e-05 2021-01-15T19:08:24.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258474+0000", 2021-01-15T19:08:24.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7099999999999998e-06 2021-01-15T19:08:24.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258479+0000", 2021-01-15T19:08:24.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0571999999999997e-05 2021-01-15T19:08:24.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258519+0000", 2021-01-15T19:08:24.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0327e-05 2021-01-15T19:08:24.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258529+0000", 2021-01-15T19:08:24.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075387700000000002 2021-01-15T19:08:24.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259283+0000", 2021-01-15T19:08:24.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0431999999999999e-05 2021-01-15T19:08:24.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259314+0000", 2021-01-15T19:08:24.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088728699999999995 2021-01-15T19:08:24.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.712 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.713 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.cs1 28/21 ECSubRead(tid=42, to_read={3:30718dd9:::smithi00140323-34:head=0,278528,0}, subchunks={3:30718dd9:::smithi00140323-34:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:24.713 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.251662+0000", 2021-01-15T19:08:24.713 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3313333979999999, 2021-01-15T19:08:24.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001207598, 2021-01-15T19:08:24.714 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.714 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.714 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251662+0000", 2021-01-15T19:08:24.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251662+0000", 2021-01-15T19:08:24.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.702e-05 2021-01-15T19:08:24.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251709+0000", 2021-01-15T19:08:24.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1551e-05 2021-01-15T19:08:24.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251721+0000", 2021-01-15T19:08:24.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9273999999999999e-05 2021-01-15T19:08:24.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251740+0000", 2021-01-15T19:08:24.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2250000000000006e-06 2021-01-15T19:08:24.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251749+0000", 2021-01-15T19:08:24.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5204999999999997e-05 2021-01-15T19:08:24.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.251814+0000", 2021-01-15T19:08:24.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010553229999999999 2021-01-15T19:08:24.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.252870+0000", 2021-01-15T19:08:24.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001207598 2021-01-15T19:08:24.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.723 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 28/21 ECSubWrite(tid=83, reqid=osd.4.0:83, at_version=28'16, trim_to=0'0, roll_forward_to=27'14))", 2021-01-15T19:08:24.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.255432+0000", 2021-01-15T19:08:24.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3275634640000007, 2021-01-15T19:08:24.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019518389999999999, 2021-01-15T19:08:24.724 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.724 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255432+0000", 2021-01-15T19:08:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255432+0000", 2021-01-15T19:08:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2109e-05 2021-01-15T19:08:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255464+0000", 2021-01-15T19:08:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6886999999999999e-05 2021-01-15T19:08:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255481+0000", 2021-01-15T19:08:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9709999999999993e-06 2021-01-15T19:08:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255490+0000", 2021-01-15T19:08:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6489999999999996e-06 2021-01-15T19:08:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255496+0000", 2021-01-15T19:08:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9673999999999998e-05 2021-01-15T19:08:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255555+0000", 2021-01-15T19:08:24.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3528e-05 2021-01-15T19:08:24.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255569+0000", 2021-01-15T19:08:24.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00175373 2021-01-15T19:08:24.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.257322+0000", 2021-01-15T19:08:24.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1291000000000007e-05 2021-01-15T19:08:24.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.257384+0000", 2021-01-15T19:08:24.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019518389999999999 2021-01-15T19:08:24.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.734 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=45, reqid=client.4418.0:204, at_version=28'16, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:24.735 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:24.735 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3275481419999995, 2021-01-15T19:08:24.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002, 2021-01-15T19:08:24.736 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.736 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.736 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:24.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:24.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1822999999999997e-05 2021-01-15T19:08:24.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255479+0000", 2021-01-15T19:08:24.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7513999999999999e-05 2021-01-15T19:08:24.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255497+0000", 2021-01-15T19:08:24.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3723000000000001e-05 2021-01-15T19:08:24.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255510+0000", 2021-01-15T19:08:24.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0780000000000003e-06 2021-01-15T19:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255515+0000", 2021-01-15T19:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2679999999999998e-05 2021-01-15T19:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255558+0000", 2021-01-15T19:08:24.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.695e-06 2021-01-15T19:08:24.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255566+0000", 2021-01-15T19:08:24.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028394660000000001 2021-01-15T19:08:24.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258405+0000", 2021-01-15T19:08:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9791999999999999e-05 2021-01-15T19:08:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258435+0000", 2021-01-15T19:08:24.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002 2021-01-15T19:08:24.745 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.746 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.746 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 28/21 ECSubWrite(tid=85, reqid=osd.4.0:83, at_version=0'0, trim_to=0'0, roll_forward_to=28'16))", 2021-01-15T19:08:24.747 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.259623+0000", 2021-01-15T19:08:24.747 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3233723860000008, 2021-01-15T19:08:24.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001392885, 2021-01-15T19:08:24.747 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.747 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.747 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259623+0000", 2021-01-15T19:08:24.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259623+0000", 2021-01-15T19:08:24.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5905999999999997e-05 2021-01-15T19:08:24.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259669+0000", 2021-01-15T19:08:24.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2064e-05 2021-01-15T19:08:24.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259681+0000", 2021-01-15T19:08:24.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1871e-05 2021-01-15T19:08:24.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259693+0000", 2021-01-15T19:08:24.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1170000000000003e-06 2021-01-15T19:08:24.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259698+0000", 2021-01-15T19:08:24.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8547000000000001e-05 2021-01-15T19:08:24.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259746+0000", 2021-01-15T19:08:24.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5950000000000004e-06 2021-01-15T19:08:24.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.259753+0000", 2021-01-15T19:08:24.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012176019999999999 2021-01-15T19:08:24.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260971+0000", 2021-01-15T19:08:24.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5182999999999998e-05 2021-01-15T19:08:24.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.261016+0000", 2021-01-15T19:08:24.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001392885 2021-01-15T19:08:24.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.757 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.758 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=47, reqid=osd.0.0:47, at_version=28'18, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:24.758 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.260417+0000", 2021-01-15T19:08:24.759 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3225778950000002, 2021-01-15T19:08:24.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017245170000000001, 2021-01-15T19:08:24.759 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.759 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.759 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260417+0000", 2021-01-15T19:08:24.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260417+0000", 2021-01-15T19:08:24.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0448000000000001e-05 2021-01-15T19:08:24.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260448+0000", 2021-01-15T19:08:24.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9599e-05 2021-01-15T19:08:24.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260468+0000", 2021-01-15T19:08:24.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1601e-05 2021-01-15T19:08:24.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260479+0000", 2021-01-15T19:08:24.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9600000000000003e-06 2021-01-15T19:08:24.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260486+0000", 2021-01-15T19:08:24.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0080999999999998e-05 2021-01-15T19:08:24.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260526+0000", 2021-01-15T19:08:24.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9719999999999998e-06 2021-01-15T19:08:24.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.260535+0000", 2021-01-15T19:08:24.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015556070000000001 2021-01-15T19:08:24.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.262091+0000", 2021-01-15T19:08:24.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1248999999999997e-05 2021-01-15T19:08:24.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.262142+0000", 2021-01-15T19:08:24.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017245170000000001 2021-01-15T19:08:24.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.769 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=48, reqid=osd.0.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=28'18))", 2021-01-15T19:08:24.770 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.265263+0000", 2021-01-15T19:08:24.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.3177328480000003, 2021-01-15T19:08:24.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096451299999999996, 2021-01-15T19:08:24.771 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.771 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265263+0000", 2021-01-15T19:08:24.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265263+0000", 2021-01-15T19:08:24.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9433000000000001e-05 2021-01-15T19:08:24.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265292+0000", 2021-01-15T19:08:24.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8470000000000008e-06 2021-01-15T19:08:24.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265302+0000", 2021-01-15T19:08:24.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0881e-05 2021-01-15T19:08:24.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265313+0000", 2021-01-15T19:08:24.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:08:24.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265319+0000", 2021-01-15T19:08:24.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5311e-05 2021-01-15T19:08:24.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265364+0000", 2021-01-15T19:08:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5469999999999999e-06 2021-01-15T19:08:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.265372+0000", 2021-01-15T19:08:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081769500000000001 2021-01-15T19:08:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.266190+0000", 2021-01-15T19:08:24.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7329999999999997e-05 2021-01-15T19:08:24.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.266227+0000", 2021-01-15T19:08:24.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096451299999999996 2021-01-15T19:08:24.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 36/21 ECSubWrite(tid=30, reqid=client.4418.0:244, at_version=36'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:24.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:24.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.29016438, 2021-01-15T19:08:24.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034199159999999998, 2021-01-15T19:08:24.783 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.783 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:24.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:24.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5223999999999999e-05 2021-01-15T19:08:24.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292876+0000", 2021-01-15T19:08:24.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.614e-05 2021-01-15T19:08:24.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292902+0000", 2021-01-15T19:08:24.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2162e-05 2021-01-15T19:08:24.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292915+0000", 2021-01-15T19:08:24.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2430000000000005e-06 2021-01-15T19:08:24.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292923+0000", 2021-01-15T19:08:24.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4569e-05 2021-01-15T19:08:24.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292967+0000", 2021-01-15T19:08:24.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2889999999999998e-06 2021-01-15T19:08:24.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292976+0000", 2021-01-15T19:08:24.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032267860000000002 2021-01-15T19:08:24.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296202+0000", 2021-01-15T19:08:24.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8503e-05 2021-01-15T19:08:24.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296251+0000", 2021-01-15T19:08:24.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034199159999999998 2021-01-15T19:08:24.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.794 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 36/21 ECSubWrite(tid=31, reqid=client.4418.0:244, at_version=0'0, trim_to=0'0, roll_forward_to=36'12))", 2021-01-15T19:08:24.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.298865+0000", 2021-01-15T19:08:24.795 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.28413040099999998, 2021-01-15T19:08:24.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002232253, 2021-01-15T19:08:24.795 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.795 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.795 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298865+0000", 2021-01-15T19:08:24.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298865+0000", 2021-01-15T19:08:24.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8621000000000001e-05 2021-01-15T19:08:24.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298914+0000", 2021-01-15T19:08:24.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5302999999999999e-05 2021-01-15T19:08:24.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298929+0000", 2021-01-15T19:08:24.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5676999999999999e-05 2021-01-15T19:08:24.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298945+0000", 2021-01-15T19:08:24.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0339999999999999e-06 2021-01-15T19:08:24.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298952+0000", 2021-01-15T19:08:24.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5219000000000001e-05 2021-01-15T19:08:24.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299007+0000", 2021-01-15T19:08:24.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3179e-05 2021-01-15T19:08:24.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299020+0000", 2021-01-15T19:08:24.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020297660000000001 2021-01-15T19:08:24.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.301050+0000", 2021-01-15T19:08:24.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7453999999999997e-05 2021-01-15T19:08:24.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.301097+0000", 2021-01-15T19:08:24.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002232253 2021-01-15T19:08:24.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.805 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.806 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.806 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/21 ECSubWrite(tid=71, reqid=client.4418.0:252, at_version=36'23, trim_to=0'0, roll_forward_to=34'21))", 2021-01-15T19:08:24.806 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:24.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27146248099999998, 2021-01-15T19:08:24.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003352748, 2021-01-15T19:08:24.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.807 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.807 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:24.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:24.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047739199999999999 2021-01-15T19:08:24.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312010+0000", 2021-01-15T19:08:24.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1661000000000001e-05 2021-01-15T19:08:24.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312032+0000", 2021-01-15T19:08:24.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1812e-05 2021-01-15T19:08:24.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312044+0000", 2021-01-15T19:08:24.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.038e-06 2021-01-15T19:08:24.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312052+0000", 2021-01-15T19:08:24.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3818999999999993e-05 2021-01-15T19:08:24.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312116+0000", 2021-01-15T19:08:24.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4571e-05 2021-01-15T19:08:24.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312130+0000", 2021-01-15T19:08:24.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002722431 2021-01-15T19:08:24.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314853+0000", 2021-01-15T19:08:24.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3024000000000001e-05 2021-01-15T19:08:24.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314886+0000", 2021-01-15T19:08:24.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003352748 2021-01-15T19:08:24.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.818 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.818 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.818 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/21 ECSubWrite(tid=72, reqid=client.4418.0:253, at_version=36'24, trim_to=0'0, roll_forward_to=34'21))", 2021-01-15T19:08:24.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.314789+0000", 2021-01-15T19:08:24.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.26820650699999998, 2021-01-15T19:08:24.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012681509999999999, 2021-01-15T19:08:24.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:24.819 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:24.820 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:24.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:24.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314789+0000", 2021-01-15T19:08:24.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:24.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:24.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314789+0000", 2021-01-15T19:08:24.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000245293 2021-01-15T19:08:24.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:24.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.315034+0000", 2021-01-15T19:08:24.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696000000000001e-05 2021-01-15T19:08:24.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:24.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.315048+0000", 2021-01-15T19:08:24.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0512e-05 2021-01-15T19:08:24.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:24.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.315058+0000", 2021-01-15T19:08:24.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1560000000000003e-06 2021-01-15T19:08:24.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:24.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.315064+0000", 2021-01-15T19:08:24.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2545000000000001e-05 2021-01-15T19:08:24.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:24.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.315096+0000", 2021-01-15T19:08:24.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6590000000000001e-06 2021-01-15T19:08:24.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:24.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.315103+0000", 2021-01-15T19:08:24.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092662899999999999 2021-01-15T19:08:24.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:24.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.316029+0000", 2021-01-15T19:08:24.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7661000000000002e-05 2021-01-15T19:08:24.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:24.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:24.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:24.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.316057+0000", 2021-01-15T19:08:24.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012681509999999999 2021-01-15T19:08:24.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:24.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:24.830 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:24.832 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:24.830+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:24.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:24.932 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:24.931+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:24.966 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:24.966 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:24.967 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:24.967 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:24.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:25.033 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:25.032+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:25.134 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:25.134+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:25.147 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:25.148 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:25.148 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:25.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:25.235 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:25.234+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:99: done (2 left) 2021-01-15T19:08:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:100: read oid 5 snap -1 2021-01-15T19:08:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:100: expect (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:08:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:101: snap_remove snap 4 2021-01-15T19:08:25.286 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:08:25.286 INFO:teuthology.orchestra.run.smithi001.stderr:set 2.1 pg_upmap_items mapping to [5->2,4->0] 2021-01-15T19:08:25.299 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:25.299 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:08:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:154 3.fs0 3:f27a3046:::smithi00140323-1:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.059016035000001, 2021-01-15T19:08:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002, 2021-01-15T19:08:25.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 154 2021-01-15T19:08:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:25.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4352999999999999e-05 2021-01-15T19:08:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237631+0000", 2021-01-15T19:08:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3304e-05 2021-01-15T19:08:25.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237644+0000", 2021-01-15T19:08:25.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1736e-05 2021-01-15T19:08:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237666+0000", 2021-01-15T19:08:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0607e-05 2021-01-15T19:08:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237676+0000", 2021-01-15T19:08:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3862e-05 2021-01-15T19:08:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237730+0000", 2021-01-15T19:08:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1849999999999998e-06 2021-01-15T19:08:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237737+0000", 2021-01-15T19:08:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014276250000000001 2021-01-15T19:08:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239165+0000", 2021-01-15T19:08:25.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7447000000000007e-05 2021-01-15T19:08:25.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239232+0000", 2021-01-15T19:08:25.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061349480000000003 2021-01-15T19:08:25.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245367+0000", 2021-01-15T19:08:25.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095382000000000002 2021-01-15T19:08:25.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254906+0000", 2021-01-15T19:08:25.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113465 2021-01-15T19:08:25.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255019+0000", 2021-01-15T19:08:25.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000454659 2021-01-15T19:08:25.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255474+0000", 2021-01-15T19:08:25.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002 2021-01-15T19:08:25.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.321 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:155 3.fs0 3:f27a3046:::smithi00140323-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:25.322 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:25.322 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.058854927000001, 2021-01-15T19:08:25.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999, 2021-01-15T19:08:25.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.323 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 155 2021-01-15T19:08:25.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.324 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:25.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:25.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9607999999999998e-05 2021-01-15T19:08:25.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237767+0000", 2021-01-15T19:08:25.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3810000000000001e-06 2021-01-15T19:08:25.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237771+0000", 2021-01-15T19:08:25.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5639999999999999e-06 2021-01-15T19:08:25.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237779+0000", 2021-01-15T19:08:25.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.348e-06 2021-01-15T19:08:25.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237782+0000", 2021-01-15T19:08:25.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3377000000000002e-05 2021-01-15T19:08:25.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237816+0000", 2021-01-15T19:08:25.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000003e-06 2021-01-15T19:08:25.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:25.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237822+0000", 2021-01-15T19:08:25.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552276 2021-01-15T19:08:25.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239374+0000", 2021-01-15T19:08:25.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:25.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:25.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239415+0000", 2021-01-15T19:08:25.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010934268 2021-01-15T19:08:25.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250349+0000", 2021-01-15T19:08:25.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011671 2021-01-15T19:08:25.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:25.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250466+0000", 2021-01-15T19:08:25.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050409820000000003 2021-01-15T19:08:25.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255507+0000", 2021-01-15T19:08:25.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0531000000000005e-05 2021-01-15T19:08:25.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255587+0000", 2021-01-15T19:08:25.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122043 2021-01-15T19:08:25.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255709+0000", 2021-01-15T19:08:25.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999 2021-01-15T19:08:25.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.342 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:156 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~442368] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:25.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:25.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.055014568000001, 2021-01-15T19:08:25.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759, 2021-01-15T19:08:25.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156 2021-01-15T19:08:25.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:25.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:25.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030551200000000003 2021-01-15T19:08:25.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241873+0000", 2021-01-15T19:08:25.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6080000000000002e-06 2021-01-15T19:08:25.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241883+0000", 2021-01-15T19:08:25.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:25.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241898+0000", 2021-01-15T19:08:25.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4010000000000002e-06 2021-01-15T19:08:25.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241904+0000", 2021-01-15T19:08:25.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6971999999999997e-05 2021-01-15T19:08:25.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241951+0000", 2021-01-15T19:08:25.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104244 2021-01-15T19:08:25.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242055+0000", 2021-01-15T19:08:25.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070015099999999999 2021-01-15T19:08:25.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242755+0000", 2021-01-15T19:08:25.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0080604460000000006 2021-01-15T19:08:25.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250816+0000", 2021-01-15T19:08:25.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00338733 2021-01-15T19:08:25.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254203+0000", 2021-01-15T19:08:25.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3444999999999998e-05 2021-01-15T19:08:25.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254287+0000", 2021-01-15T19:08:25.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759 2021-01-15T19:08:25.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.360 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:157 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~401408] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:25.360 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:25.360 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.052208341, 2021-01-15T19:08:25.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335, 2021-01-15T19:08:25.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 157 2021-01-15T19:08:25.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.362 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:25.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:25.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000301899 2021-01-15T19:08:25.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244676+0000", 2021-01-15T19:08:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6430000000000001e-06 2021-01-15T19:08:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244685+0000", 2021-01-15T19:08:25.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5293000000000001e-05 2021-01-15T19:08:25.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244700+0000", 2021-01-15T19:08:25.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6499999999999996e-06 2021-01-15T19:08:25.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244708+0000", 2021-01-15T19:08:25.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1064999999999997e-05 2021-01-15T19:08:25.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244789+0000", 2021-01-15T19:08:25.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117805 2021-01-15T19:08:25.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244906+0000", 2021-01-15T19:08:25.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070339100000000004 2021-01-15T19:08:25.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245610+0000", 2021-01-15T19:08:25.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087607599999999994 2021-01-15T19:08:25.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:25.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107695 2021-01-15T19:08:25.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254478+0000", 2021-01-15T19:08:25.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109134 2021-01-15T19:08:25.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254587+0000", 2021-01-15T19:08:25.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335 2021-01-15T19:08:25.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.376 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.377 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:158 3.cs0 3:3ef45b66:::smithi00140323-11:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:25.377 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:25.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.051822324, 2021-01-15T19:08:25.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441, 2021-01-15T19:08:25.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 158 2021-01-15T19:08:25.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:25.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:25.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5699999999999999e-05 2021-01-15T19:08:25.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244816+0000", 2021-01-15T19:08:25.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6839999999999996e-06 2021-01-15T19:08:25.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244823+0000", 2021-01-15T19:08:25.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4626e-05 2021-01-15T19:08:25.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244838+0000", 2021-01-15T19:08:25.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6500000000000001e-06 2021-01-15T19:08:25.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244844+0000", 2021-01-15T19:08:25.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004205149 2021-01-15T19:08:25.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249049+0000", 2021-01-15T19:08:25.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015840599999999999 2021-01-15T19:08:25.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249207+0000", 2021-01-15T19:08:25.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071027700000000004 2021-01-15T19:08:25.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249918+0000", 2021-01-15T19:08:25.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051411900000000003 2021-01-15T19:08:25.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255059+0000", 2021-01-15T19:08:25.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045701699999999999 2021-01-15T19:08:25.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255516+0000", 2021-01-15T19:08:25.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8710999999999997e-05 2021-01-15T19:08:25.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255614+0000", 2021-01-15T19:08:25.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441 2021-01-15T19:08:25.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.395 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:159 3.cs0 3:3ef45b66:::smithi00140323-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:25.395 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:25.395 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.051689873000001, 2021-01-15T19:08:25.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195, 2021-01-15T19:08:25.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 159 2021-01-15T19:08:25.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.397 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:25.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:25.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0961000000000003e-05 2021-01-15T19:08:25.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244933+0000", 2021-01-15T19:08:25.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0480000000000001e-06 2021-01-15T19:08:25.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244941+0000", 2021-01-15T19:08:25.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3711e-05 2021-01-15T19:08:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244954+0000", 2021-01-15T19:08:25.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2390000000000002e-06 2021-01-15T19:08:25.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244959+0000", 2021-01-15T19:08:25.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082968859999999998 2021-01-15T19:08:25.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253256+0000", 2021-01-15T19:08:25.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000147608 2021-01-15T19:08:25.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:25.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253404+0000", 2021-01-15T19:08:25.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047189329999999998 2021-01-15T19:08:25.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258123+0000", 2021-01-15T19:08:25.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010107000000000001 2021-01-15T19:08:25.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258224+0000", 2021-01-15T19:08:25.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019277389999999999 2021-01-15T19:08:25.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260152+0000", 2021-01-15T19:08:25.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195 2021-01-15T19:08:25.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.2s2 26/21 ECSubWrite(tid=60, reqid=client.4418.0:166, at_version=26'8, trim_to=0'0, roll_forward_to=25'6))", 2021-01-15T19:08:25.412 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:25.412 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.043739771, 2021-01-15T19:08:25.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053879080000000003, 2021-01-15T19:08:25.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:25.413 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:25.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.252843+0000", 2021-01-15T19:08:25.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000255274 2021-01-15T19:08:25.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253098+0000", 2021-01-15T19:08:25.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6628999999999997e-05 2021-01-15T19:08:25.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253145+0000", 2021-01-15T19:08:25.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7581e-05 2021-01-15T19:08:25.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253162+0000", 2021-01-15T19:08:25.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3610000000000003e-06 2021-01-15T19:08:25.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253169+0000", 2021-01-15T19:08:25.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2859000000000003e-05 2021-01-15T19:08:25.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253222+0000", 2021-01-15T19:08:25.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2021000000000001e-05 2021-01-15T19:08:25.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253234+0000", 2021-01-15T19:08:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049458130000000003 2021-01-15T19:08:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258180+0000", 2021-01-15T19:08:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0370000000000001e-05 2021-01-15T19:08:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258231+0000", 2021-01-15T19:08:25.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0053879080000000003 2021-01-15T19:08:25.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.425 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.425 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:201 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:25.426 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:25.426 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.0483365459999998, 2021-01-15T19:08:25.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142, 2021-01-15T19:08:25.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.427 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 201 2021-01-15T19:08:25.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.428 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:25.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:25.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1060000000000002e-05 2021-01-15T19:08:25.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248307+0000", 2021-01-15T19:08:25.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6740000000000003e-06 2021-01-15T19:08:25.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248316+0000", 2021-01-15T19:08:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8746e-05 2021-01-15T19:08:25.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248334+0000", 2021-01-15T19:08:25.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2294e-05 2021-01-15T19:08:25.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248347+0000", 2021-01-15T19:08:25.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6400999999999998e-05 2021-01-15T19:08:25.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248393+0000", 2021-01-15T19:08:25.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2021-01-15T19:08:25.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:25.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248400+0000", 2021-01-15T19:08:25.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002761197 2021-01-15T19:08:25.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251161+0000", 2021-01-15T19:08:25.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011450099999999999 2021-01-15T19:08:25.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251275+0000", 2021-01-15T19:08:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062315829999999997 2021-01-15T19:08:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257507+0000", 2021-01-15T19:08:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142 2021-01-15T19:08:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:202 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.0481831709999998, 2021-01-15T19:08:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285, 2021-01-15T19:08:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 202 2021-01-15T19:08:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5448999999999997e-05 2021-01-15T19:08:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248455+0000", 2021-01-15T19:08:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248461+0000", 2021-01-15T19:08:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4307e-05 2021-01-15T19:08:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248476+0000", 2021-01-15T19:08:25.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5819999999999998e-06 2021-01-15T19:08:25.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248482+0000", 2021-01-15T19:08:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1987e-05 2021-01-15T19:08:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248564+0000", 2021-01-15T19:08:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5850000000000002e-06 2021-01-15T19:08:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248572+0000", 2021-01-15T19:08:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045194099999999997 2021-01-15T19:08:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253091+0000", 2021-01-15T19:08:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020063399999999999 2021-01-15T19:08:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253292+0000", 2021-01-15T19:08:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079735020000000004 2021-01-15T19:08:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261266+0000", 2021-01-15T19:08:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285 2021-01-15T19:08:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:203 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.0480041530000008, 2021-01-15T19:08:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438, 2021-01-15T19:08:25.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.459 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 203 2021-01-15T19:08:25.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.460 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8518999999999999e-05 2021-01-15T19:08:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248607+0000", 2021-01-15T19:08:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8810000000000004e-06 2021-01-15T19:08:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248612+0000", 2021-01-15T19:08:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5719999999999997e-06 2021-01-15T19:08:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248619+0000", 2021-01-15T19:08:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5899999999999999e-06 2021-01-15T19:08:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248623+0000", 2021-01-15T19:08:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5608000000000001e-05 2021-01-15T19:08:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248668+0000", 2021-01-15T19:08:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0166000000000001e-05 2021-01-15T19:08:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248679+0000", 2021-01-15T19:08:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048830590000000004 2021-01-15T19:08:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253562+0000", 2021-01-15T19:08:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7e-05 2021-01-15T19:08:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253659+0000", 2021-01-15T19:08:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081539850000000007 2021-01-15T19:08:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261813+0000", 2021-01-15T19:08:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438 2021-01-15T19:08:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:204 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat,delete] snapc 3=[3] ondisk+write+known_if_redirected e28)", 2021-01-15T19:08:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.0479199460000004, 2021-01-15T19:08:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532, 2021-01-15T19:08:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 204 2021-01-15T19:08:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8349000000000002e-05 2021-01-15T19:08:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248701+0000", 2021-01-15T19:08:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0620000000000001e-06 2021-01-15T19:08:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248709+0000", 2021-01-15T19:08:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7700000000000001e-06 2021-01-15T19:08:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248717+0000", 2021-01-15T19:08:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5920000000000001e-06 2021-01-15T19:08:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248720+0000", 2021-01-15T19:08:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0898e-05 2021-01-15T19:08:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248761+0000", 2021-01-15T19:08:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-06 2021-01-15T19:08:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248770+0000", 2021-01-15T19:08:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052185720000000003 2021-01-15T19:08:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253989+0000", 2021-01-15T19:08:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105809 2021-01-15T19:08:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254095+0000", 2021-01-15T19:08:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078837199999999997 2021-01-15T19:08:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254883+0000", 2021-01-15T19:08:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083722769999999992 2021-01-15T19:08:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263255+0000", 2021-01-15T19:08:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111839 2021-01-15T19:08:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263367+0000", 2021-01-15T19:08:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015969899999999999 2021-01-15T19:08:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263527+0000", 2021-01-15T19:08:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532 2021-01-15T19:08:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpReadReply(3.cs0 28/21 ECSubReadReply(tid=43, attrs_read=0))", 2021-01-15T19:08:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.0395810310000009, 2021-01-15T19:08:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065310519999999999, 2021-01-15T19:08:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:08:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257001+0000", 2021-01-15T19:08:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001039442 2021-01-15T19:08:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.258041+0000", 2021-01-15T19:08:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.784e-06 2021-01-15T19:08:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.258047+0000", 2021-01-15T19:08:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026103659999999998 2021-01-15T19:08:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.260657+0000", 2021-01-15T19:08:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028754599999999998 2021-01-15T19:08:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263532+0000", 2021-01-15T19:08:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065310519999999999 2021-01-15T19:08:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:245 3.fs0 3:fae92f17:::smithi00140323-19:head [create,setxattr YOk\\Qsqnjk`Uh[icZasWhcZnad[h[k[Qmk]TihbR (1823) in=40b,setxattr QnfshqY (34) in=7b,setxattr V_tZOrZWhpPekPNs_toQaraRlW\\sWffpapfPROXb (1614) in=40b,setxattr aiTbV]hXcpXjkS]VQonZZkfogbUSnO[bajbZWdRf (839) in=40b,setxattr pZdlecZrV]qp]`sTkf_bSjZVZjhbScXfb[Sr_bsa (18524) in=40b,setxattr kXtOdjofgYOa_Tck\\nYsbVOfgtldphnQ`]P[j]lX (1295) in=40b,setxattr dRpscRhc^OtlbThWSmXVSNge]s (34) in=26b,setxattr T\\s[QdNhlRZh\\\\[X]NpSccmfRcqRaXScZOZ\\ZR^e (1999) in=40b,setxattr YTcfnqlbrRa^rhctrncU_Wa]laamPmYsaqg_f^N` (1471) in=40b,setxattr l_Qeh\\RcafrRYN_YZf_iiaWagZfe[sop]^\\qTggq (561) in=40b,setxattr NShVOQe\\dRYY[tPettNbkgkTVonNshnbYeePWcla (314) in=40b,setxattr ^iVidVQX]Yn`XTY\\pOXm`_PNX`qeckhbQoR`aQWZ (8841) in=40b,setxattr _jkmO]cXrN_S^act\\QOOjpUZOj_Qf^otnZbpWZPU (981) in=40b,setxattr bNbhPX[YP[lppmc`PkWcpnS`XsQNpW\\UYnWr\\_RX (1927) in=40b,setxattr talTebhpYS[[TXcgodoQR\\tf`fVekppjlkWnTqf[ (1753) in=40b,setxattr rp\\XTgd\\X^qaodkhY[elq^RagRhN_NR^j^NmjOUh (806) in=40b,setxattr ]\\oXSo`PWafUO^dbhR^i`hifQgWjRhR[rRa^bnej (70) in=40b,setxattr UkYrdYaZakpnReakOdc\\^bO[YPqfdkdtbNkghaSQ (279) in=40b,setxattr YtUs_hcRdQqgZV]NaadOflcb\\_c^\\Zg_Y\\]snXXZ (701) in=40b,setxattr NRZdh]WclahTe^_btdqe\\^mVYqQOaXaa\\aPVXWnk (19016) in=40b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.99608570500000004, 2021-01-15T19:08:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083409910000000007, 2021-01-15T19:08:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 245 2021-01-15T19:08:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122779 2021-01-15T19:08:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300619+0000", 2021-01-15T19:08:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0987e-05 2021-01-15T19:08:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300630+0000", 2021-01-15T19:08:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0562000000000001e-05 2021-01-15T19:08:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300651+0000", 2021-01-15T19:08:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329999999999999e-05 2021-01-15T19:08:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300661+0000", 2021-01-15T19:08:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6162000000000002e-05 2021-01-15T19:08:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300718+0000", 2021-01-15T19:08:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105159 2021-01-15T19:08:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300823+0000", 2021-01-15T19:08:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011057879999999999 2021-01-15T19:08:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301928+0000", 2021-01-15T19:08:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064543059999999999 2021-01-15T19:08:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308383+0000", 2021-01-15T19:08:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031878700000000001 2021-01-15T19:08:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308702+0000", 2021-01-15T19:08:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136131 2021-01-15T19:08:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308838+0000", 2021-01-15T19:08:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083409910000000007 2021-01-15T19:08:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:246 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.99587113400000005, 2021-01-15T19:08:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005, 2021-01-15T19:08:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 246 2021-01-15T19:08:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1804000000000001e-05 2021-01-15T19:08:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300763+0000", 2021-01-15T19:08:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5610000000000004e-06 2021-01-15T19:08:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300770+0000", 2021-01-15T19:08:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300782+0000", 2021-01-15T19:08:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4460000000000003e-06 2021-01-15T19:08:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300786+0000", 2021-01-15T19:08:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003399596 2021-01-15T19:08:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304186+0000", 2021-01-15T19:08:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125399 2021-01-15T19:08:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304311+0000", 2021-01-15T19:08:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057843119999999998 2021-01-15T19:08:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310095+0000", 2021-01-15T19:08:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005 2021-01-15T19:08:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:247 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.99575402099999999, 2021-01-15T19:08:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281, 2021-01-15T19:08:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 247 2021-01-15T19:08:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2098999999999997e-05 2021-01-15T19:08:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300880+0000", 2021-01-15T19:08:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2021-01-15T19:08:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300887+0000", 2021-01-15T19:08:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.152e-05 2021-01-15T19:08:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300898+0000", 2021-01-15T19:08:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.904e-06 2021-01-15T19:08:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300903+0000", 2021-01-15T19:08:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039909860000000002 2021-01-15T19:08:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304894+0000", 2021-01-15T19:08:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012664060000000001 2021-01-15T19:08:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306161+0000", 2021-01-15T19:08:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044999549999999999 2021-01-15T19:08:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310661+0000", 2021-01-15T19:08:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281 2021-01-15T19:08:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:248 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.99563949299999999, 2021-01-15T19:08:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004, 2021-01-15T19:08:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 248 2021-01-15T19:08:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9626999999999998e-05 2021-01-15T19:08:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300993+0000", 2021-01-15T19:08:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1990000000000003e-06 2021-01-15T19:08:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300999+0000", 2021-01-15T19:08:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1737e-05 2021-01-15T19:08:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301010+0000", 2021-01-15T19:08:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.006e-05 2021-01-15T19:08:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301021+0000", 2021-01-15T19:08:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054558510000000003 2021-01-15T19:08:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306476+0000", 2021-01-15T19:08:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000193029 2021-01-15T19:08:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306669+0000", 2021-01-15T19:08:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046625010000000003 2021-01-15T19:08:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.311332+0000", 2021-01-15T19:08:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004 2021-01-15T19:08:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:252 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~647168] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98790106300000002, 2021-01-15T19:08:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996, 2021-01-15T19:08:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 252 2021-01-15T19:08:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041048799999999998 2021-01-15T19:08:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309092+0000", 2021-01-15T19:08:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:08:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309101+0000", 2021-01-15T19:08:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4919000000000001e-05 2021-01-15T19:08:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309116+0000", 2021-01-15T19:08:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2021-01-15T19:08:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309122+0000", 2021-01-15T19:08:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0148999999999997e-05 2021-01-15T19:08:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309172+0000", 2021-01-15T19:08:25.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013501799999999999 2021-01-15T19:08:25.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309307+0000", 2021-01-15T19:08:25.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015785269999999999 2021-01-15T19:08:25.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310886+0000", 2021-01-15T19:08:25.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070233450000000003 2021-01-15T19:08:25.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.317909+0000", 2021-01-15T19:08:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2075000000000002e-05 2021-01-15T19:08:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318001+0000", 2021-01-15T19:08:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0710999999999994e-05 2021-01-15T19:08:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318072+0000", 2021-01-15T19:08:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996 2021-01-15T19:08:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:253 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.312902+0000", 2021-01-15T19:08:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98368003199999998, 2021-01-15T19:08:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052577910000000004, 2021-01-15T19:08:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 253 2021-01-15T19:08:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.312902+0000", 2021-01-15T19:08:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.312902+0000", 2021-01-15T19:08:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017438999999999999 2021-01-15T19:08:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313077+0000", 2021-01-15T19:08:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4930000000000004e-06 2021-01-15T19:08:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313086+0000", 2021-01-15T19:08:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5943999999999998e-05 2021-01-15T19:08:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313102+0000", 2021-01-15T19:08:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:08:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313110+0000", 2021-01-15T19:08:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00096367499999999999 2021-01-15T19:08:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.314074+0000", 2021-01-15T19:08:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4889000000000007e-05 2021-01-15T19:08:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.314159+0000", 2021-01-15T19:08:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00047565500000000001 2021-01-15T19:08:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.314634+0000", 2021-01-15T19:08:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0034399370000000001 2021-01-15T19:08:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318074+0000", 2021-01-15T19:08:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4787999999999999e-05 2021-01-15T19:08:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318119+0000", 2021-01-15T19:08:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1010000000000002e-05 2021-01-15T19:08:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318160+0000", 2021-01-15T19:08:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052577910000000004 2021-01-15T19:08:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:254 3.cs0 3:30718dd9:::smithi00140323-34:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.313161+0000", 2021-01-15T19:08:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98342093600000002, 2021-01-15T19:08:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005333183, 2021-01-15T19:08:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 254 2021-01-15T19:08:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313161+0000", 2021-01-15T19:08:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313161+0000", 2021-01-15T19:08:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7456e-05 2021-01-15T19:08:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313219+0000", 2021-01-15T19:08:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9959999999999996e-06 2021-01-15T19:08:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313226+0000", 2021-01-15T19:08:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2531000000000001e-05 2021-01-15T19:08:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313238+0000", 2021-01-15T19:08:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0180000000000001e-06 2021-01-15T19:08:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313243+0000", 2021-01-15T19:08:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025815259999999998 2021-01-15T19:08:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.315825+0000", 2021-01-15T19:08:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111508 2021-01-15T19:08:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.315936+0000", 2021-01-15T19:08:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000453296 2021-01-15T19:08:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.316390+0000", 2021-01-15T19:08:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001772144 2021-01-15T19:08:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318162+0000", 2021-01-15T19:08:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9966999999999999e-05 2021-01-15T19:08:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318202+0000", 2021-01-15T19:08:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029274100000000001 2021-01-15T19:08:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318495+0000", 2021-01-15T19:08:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005333183 2021-01-15T19:08:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:255 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.313285+0000", 2021-01-15T19:08:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98329723700000005, 2021-01-15T19:08:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062077089999999996, 2021-01-15T19:08:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 255 2021-01-15T19:08:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313285+0000", 2021-01-15T19:08:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313285+0000", 2021-01-15T19:08:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8228000000000001e-05 2021-01-15T19:08:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313323+0000", 2021-01-15T19:08:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.195e-06 2021-01-15T19:08:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313330+0000", 2021-01-15T19:08:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2275e-05 2021-01-15T19:08:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313342+0000", 2021-01-15T19:08:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0630000000000002e-06 2021-01-15T19:08:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.313348+0000", 2021-01-15T19:08:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032605450000000001 2021-01-15T19:08:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.316608+0000", 2021-01-15T19:08:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114036 2021-01-15T19:08:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.316722+0000", 2021-01-15T19:08:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018072590000000001 2021-01-15T19:08:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318530+0000", 2021-01-15T19:08:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011038400000000001 2021-01-15T19:08:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318640+0000", 2021-01-15T19:08:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085272400000000002 2021-01-15T19:08:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.319493+0000", 2021-01-15T19:08:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062077089999999996 2021-01-15T19:08:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:25.633 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:25.633 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:25.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:25.636 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:25.633+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:25.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:25.735 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:25.734+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:25.813 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:25.814 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:25.814 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:25.824 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:25.827 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:25.848 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:25.848 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:25.847+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:25.869 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:25.889 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:25.910 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:25.931 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:25.936 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:25.935+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:25.952 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:25.959 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:25.960 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:25.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:25.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:25.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:25.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:26.037 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:26.035+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:26.104 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:26.104 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:26.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:26.138 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:26.136+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:26.239 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:26.238+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:26.247 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:26.248 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:26.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:26.285 INFO:tasks.rados.rados.0.smithi001.stdout:101: done (3 left) 2021-01-15T19:08:26.286 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 40 current snap is 7 2021-01-15T19:08:26.286 INFO:tasks.rados.rados.0.smithi001.stdout:102: seq_num 63 ranges {1679360=663552,2342912=401408} 2021-01-15T19:08:26.298 INFO:tasks.rados.rados.0.smithi001.stdout:102: writing smithi00140323-40 from 1679360 to 2342912 tid 1 2021-01-15T19:08:26.300 INFO:tasks.rados.rados.0.smithi001.stdout:102: writing smithi00140323-40 from 2342912 to 2744320 tid 2 2021-01-15T19:08:26.300 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 7 2021-01-15T19:08:26.301 INFO:tasks.rados.rados.0.smithi001.stdout:96: expect (ObjNum 35 snap 0 seq_num 35) 2021-01-15T19:08:26.317 INFO:tasks.rados.rados.0.smithi001.stdout:98: finishing write tid 1 to smithi00140323-34 2021-01-15T19:08:26.317 INFO:tasks.rados.rados.0.smithi001.stdout:98: finishing write tid 2 to smithi00140323-34 2021-01-15T19:08:26.317 INFO:tasks.rados.rados.0.smithi001.stdout:98: finishing write tid 3 to smithi00140323-34 2021-01-15T19:08:26.317 INFO:tasks.rados.rados.0.smithi001.stdout:98: finishing write tid 4 to smithi00140323-34 2021-01-15T19:08:26.318 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 34 v 25 (ObjNum 62 snap 7 seq_num 62) dirty exists 2021-01-15T19:08:26.318 INFO:tasks.rados.rados.0.smithi001.stdout:98: left oid 34 (ObjNum 62 snap 7 seq_num 62) 2021-01-15T19:08:26.318 INFO:tasks.rados.rados.0.smithi001.stdout:103: seq_num 64 ranges {679936=679936,1359872=401408} 2021-01-15T19:08:26.324 INFO:tasks.rados.rados.0.smithi001.stdout:103: writing smithi00140323-39 from 679936 to 1359872 tid 1 2021-01-15T19:08:26.326 INFO:tasks.rados.rados.0.smithi001.stdout:103: writing smithi00140323-39 from 1359872 to 1761280 tid 2 2021-01-15T19:08:26.327 INFO:tasks.rados.rados.0.smithi001.stdout:96: done (4 left) 2021-01-15T19:08:26.327 INFO:tasks.rados.rados.0.smithi001.stdout:98: done (3 left) 2021-01-15T19:08:26.327 INFO:tasks.rados.rados.0.smithi001.stdout:104: copy_from oid 16 from oid 3 current snap is 7 2021-01-15T19:08:26.327 INFO:tasks.rados.rados.0.smithi001.stdout:100: expect (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:08:26.329 INFO:tasks.rados.rados.0.smithi001.stdout:102: finishing write tid 1 to smithi00140323-40 2021-01-15T19:08:26.329 INFO:tasks.rados.rados.0.smithi001.stdout:102: finishing write tid 2 to smithi00140323-40 2021-01-15T19:08:26.329 INFO:tasks.rados.rados.0.smithi001.stdout:102: finishing write tid 3 to smithi00140323-40 2021-01-15T19:08:26.329 INFO:tasks.rados.rados.0.smithi001.stdout:102: finishing write tid 4 to smithi00140323-40 2021-01-15T19:08:26.330 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 40 v 21 (ObjNum 63 snap 7 seq_num 63) dirty exists 2021-01-15T19:08:26.330 INFO:tasks.rados.rados.0.smithi001.stdout:102: left oid 40 (ObjNum 63 snap 7 seq_num 63) 2021-01-15T19:08:26.330 INFO:tasks.rados.rados.0.smithi001.stdout:100: done (3 left) 2021-01-15T19:08:26.330 INFO:tasks.rados.rados.0.smithi001.stdout:102: done (2 left) 2021-01-15T19:08:26.330 INFO:tasks.rados.rados.0.smithi001.stdout:105: copy_from oid 12 from oid 5 current snap is 7 2021-01-15T19:08:26.331 INFO:tasks.rados.rados.0.smithi001.stdout:106: read oid 40 snap -1 2021-01-15T19:08:26.331 INFO:tasks.rados.rados.0.smithi001.stdout:106: expect (ObjNum 63 snap 7 seq_num 63) 2021-01-15T19:08:26.331 INFO:tasks.rados.rados.0.smithi001.stdout:107: copy_from oid 32 from oid 44 current snap is 7 2021-01-15T19:08:26.332 INFO:tasks.rados.rados.0.smithi001.stdout:108: snap_remove snap 1 2021-01-15T19:08:26.342 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:26.338+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:26.426 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:26.427 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:26.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:26.478 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:26.476+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:26.542 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:26.540+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:26.607 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:26.608 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:26.619 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:26.660 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:26.658+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:26.746 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:26.744+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:26.793 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:26.794 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:26.808 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:26.847 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:26.844+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:26.948 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:26.946+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:26.985 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:26.985 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:26.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:27.048 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:27.047+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:27.126 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:27.127 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:27.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:27.177 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:27.176+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:27.250 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:27.249+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:27.274 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:27.274 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:27.283 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:27.291 INFO:tasks.rados.rados.0.smithi001.stdout:108: done (5 left) 2021-01-15T19:08:27.292 INFO:tasks.rados.rados.0.smithi001.stdout:109: read oid 23 snap -1 2021-01-15T19:08:27.293 INFO:tasks.rados.rados.0.smithi001.stdout:109: expect (ObjNum 22 snap 0 seq_num 22) 2021-01-15T19:08:27.293 INFO:tasks.rados.rados.0.smithi001.stdout:110: copy_from oid 11 from oid 20 current snap is 7 2021-01-15T19:08:27.293 INFO:tasks.rados.rados.0.smithi001.stdout:111: snap_create 2021-01-15T19:08:27.294 INFO:tasks.rados.rados.0.smithi001.stdout:103: finishing write tid 1 to smithi00140323-39 2021-01-15T19:08:27.295 INFO:tasks.rados.rados.0.smithi001.stdout:103: finishing write tid 2 to smithi00140323-39 2021-01-15T19:08:27.295 INFO:tasks.rados.rados.0.smithi001.stdout:103: finishing write tid 3 to smithi00140323-39 2021-01-15T19:08:27.295 INFO:tasks.rados.rados.0.smithi001.stdout:104: finishing copy_from to smithi00140323-16 2021-01-15T19:08:27.296 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2021-01-15T19:08:27.296 INFO:tasks.rados.rados.0.smithi001.stdout:104: finishing copy_from racing read to smithi00140323-16 2021-01-15T19:08:27.297 INFO:tasks.rados.rados.0.smithi001.stdout:105: finishing copy_from to smithi00140323-12 2021-01-15T19:08:27.297 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 4 (ObjNum 4 snap 0 seq_num 4) dirty exists 2021-01-15T19:08:27.298 INFO:tasks.rados.rados.0.smithi001.stdout:103: finishing write tid 4 to smithi00140323-39 2021-01-15T19:08:27.298 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 19 (ObjNum 64 snap 7 seq_num 64) dirty exists 2021-01-15T19:08:27.299 INFO:tasks.rados.rados.0.smithi001.stdout:103: left oid 39 (ObjNum 64 snap 7 seq_num 64) 2021-01-15T19:08:27.299 INFO:tasks.rados.rados.0.smithi001.stdout:105: finishing copy_from racing read to smithi00140323-12 2021-01-15T19:08:27.300 INFO:tasks.rados.rados.0.smithi001.stdout:107: finishing copy_from to smithi00140323-32 2021-01-15T19:08:27.300 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 4 (ObjNum 43 snap 0 seq_num 43) dirty exists 2021-01-15T19:08:27.301 INFO:tasks.rados.rados.0.smithi001.stdout:107: finishing copy_from racing read to smithi00140323-32 2021-01-15T19:08:27.301 INFO:tasks.rados.rados.0.smithi001.stdout:106: expect (ObjNum 63 snap 7 seq_num 63) 2021-01-15T19:08:27.302 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:27.294+0000 7f0f4a898700 -1 osd.5 pg_epoch: 39 pg[3.0s0( v 39'20 (0'0,39'20] local-lis/les=21/22 n=6 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) [5,7,2]p5(0) r=0 lpr=21 luod=38'18 crt=38'18 lcod 38'16 mlcod 38'16 active+clean+snaptrim trimq=[3~1] ps=[2~1,5~3,9~1,c~1]] removing snap head 2021-01-15T19:08:27.350 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:27.348+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:27.407 INFO:tasks.rados.rados.0.smithi001.stdout:109: expect (ObjNum 22 snap 0 seq_num 22) 2021-01-15T19:08:27.416 INFO:tasks.rados.rados.0.smithi001.stdout:110: finishing copy_from to smithi00140323-11 2021-01-15T19:08:27.417 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 8 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:08:27.417 INFO:tasks.rados.rados.0.smithi001.stdout:110: finishing copy_from racing read to smithi00140323-11 2021-01-15T19:08:27.418 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:27.419 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:27.426 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:27.467 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:27.465+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:27.552 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:27.550+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:27.590 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:27.591 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:27.600 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:27.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:08:27.682 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:27.680+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:27.753 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:27.752+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:27.765 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:27.765 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:27.775 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:27.855 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:27.852+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:27.901 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:27.902 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:27.911 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:27.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:27.994 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:27.992+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:28.056 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:28.053+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:28.075 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:28.076 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:28.085 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:28.134 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:28.154 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:28.169 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:28.167+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:28.174 INFO:tasks.rados.rados.0.smithi001.stdout:103: done (7 left) 2021-01-15T19:08:28.175 INFO:tasks.rados.rados.0.smithi001.stdout:104: done (6 left) 2021-01-15T19:08:28.175 INFO:tasks.rados.rados.0.smithi001.stdout:105: done (5 left) 2021-01-15T19:08:28.175 INFO:tasks.rados.rados.0.smithi001.stdout:106: done (4 left) 2021-01-15T19:08:28.175 INFO:tasks.rados.rados.0.smithi001.stdout:107: done (3 left) 2021-01-15T19:08:28.176 INFO:tasks.rados.rados.0.smithi001.stdout:109: done (2 left) 2021-01-15T19:08:28.176 INFO:tasks.rados.rados.0.smithi001.stdout:110: done (1 left) 2021-01-15T19:08:28.177 INFO:tasks.rados.rados.0.smithi001.stdout:111: done (0 left) 2021-01-15T19:08:28.177 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 34 current snap is 8 2021-01-15T19:08:28.177 INFO:tasks.rados.rados.0.smithi001.stdout:112: seq_num 65 ranges {1605632=696320,2301952=401408} 2021-01-15T19:08:28.177 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:08:28.178 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:28.187 INFO:tasks.rados.rados.0.smithi001.stdout:112: writing smithi00140323-34 from 1605632 to 2301952 tid 1 2021-01-15T19:08:28.190 INFO:tasks.rados.rados.0.smithi001.stdout:112: writing smithi00140323-34 from 2301952 to 2703360 tid 2 2021-01-15T19:08:28.190 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 26 current snap is 8 2021-01-15T19:08:28.190 INFO:tasks.rados.rados.0.smithi001.stdout:113: seq_num 66 ranges {425984=712704,1138688=401408} 2021-01-15T19:08:28.197 INFO:tasks.rados.rados.0.smithi001.stdout:113: writing smithi00140323-26 from 425984 to 1138688 tid 1 2021-01-15T19:08:28.199 INFO:tasks.rados.rados.0.smithi001.stdout:113: writing smithi00140323-26 from 1138688 to 1540096 tid 2 2021-01-15T19:08:28.200 INFO:tasks.rados.rados.0.smithi001.stdout:114: read oid 2 snap 7 2021-01-15T19:08:28.200 INFO:tasks.rados.rados.0.smithi001.stdout:114: expect (ObjNum 1 snap 0 seq_num 1) 2021-01-15T19:08:28.200 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 19 current snap is 8 2021-01-15T19:08:28.200 INFO:tasks.rados.rados.0.smithi001.stdout:112: finishing write tid 1 to smithi00140323-34 2021-01-15T19:08:28.201 INFO:tasks.rados.rados.0.smithi001.stdout:115: seq_num 67 ranges {1261568=729088,1990656=401408} 2021-01-15T19:08:28.208 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:28.212 INFO:tasks.rados.rados.0.smithi001.stdout:115: writing smithi00140323-19 from 1261568 to 1990656 tid 1 2021-01-15T19:08:28.215 INFO:tasks.rados.rados.0.smithi001.stdout:115: writing smithi00140323-19 from 1990656 to 2392064 tid 2 2021-01-15T19:08:28.215 INFO:tasks.rados.rados.0.smithi001.stdout:112: finishing write tid 2 to smithi00140323-34 2021-01-15T19:08:28.215 INFO:tasks.rados.rados.0.smithi001.stdout:112: finishing write tid 3 to smithi00140323-34 2021-01-15T19:08:28.216 INFO:tasks.rados.rados.0.smithi001.stdout:112: finishing write tid 4 to smithi00140323-34 2021-01-15T19:08:28.216 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 34 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:08:28.216 INFO:tasks.rados.rados.0.smithi001.stdout:112: left oid 34 (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:08:28.216 INFO:tasks.rados.rados.0.smithi001.stdout:114: expect (ObjNum 1 snap 0 seq_num 1) 2021-01-15T19:08:28.217 INFO:tasks.rados.rados.0.smithi001.stdout:113: finishing write tid 1 to smithi00140323-26 2021-01-15T19:08:28.217 INFO:tasks.rados.rados.0.smithi001.stdout:113: finishing write tid 2 to smithi00140323-26 2021-01-15T19:08:28.218 INFO:tasks.rados.rados.0.smithi001.stdout:113: finishing write tid 3 to smithi00140323-26 2021-01-15T19:08:28.218 INFO:tasks.rados.rados.0.smithi001.stdout:113: finishing write tid 4 to smithi00140323-26 2021-01-15T19:08:28.218 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 22 (ObjNum 66 snap 8 seq_num 66) dirty exists 2021-01-15T19:08:28.219 INFO:tasks.rados.rados.0.smithi001.stdout:113: left oid 26 (ObjNum 66 snap 8 seq_num 66) 2021-01-15T19:08:28.219 INFO:tasks.rados.rados.0.smithi001.stdout:112: done (3 left) 2021-01-15T19:08:28.219 INFO:tasks.rados.rados.0.smithi001.stdout:113: done (2 left) 2021-01-15T19:08:28.219 INFO:tasks.rados.rados.0.smithi001.stdout:114: done (1 left) 2021-01-15T19:08:28.219 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 26 current snap is 8 2021-01-15T19:08:28.220 INFO:tasks.rados.rados.0.smithi001.stdout:116: seq_num 68 ranges {1540096=745472,2285568=401408} 2021-01-15T19:08:28.231 INFO:tasks.rados.rados.0.smithi001.stdout:116: writing smithi00140323-26 from 1540096 to 2285568 tid 1 2021-01-15T19:08:28.234 INFO:tasks.rados.rados.0.smithi001.stdout:116: writing smithi00140323-26 from 2285568 to 2686976 tid 2 2021-01-15T19:08:28.234 INFO:tasks.rados.rados.0.smithi001.stdout:117: setattr oid 21 current snap is 8 2021-01-15T19:08:28.235 INFO:tasks.rados.rados.0.smithi001.stdout:115: finishing write tid 1 to smithi00140323-19 2021-01-15T19:08:28.235 INFO:tasks.rados.rados.0.smithi001.stdout:115: finishing write tid 2 to smithi00140323-19 2021-01-15T19:08:28.235 INFO:tasks.rados.rados.0.smithi001.stdout:115: finishing write tid 3 to smithi00140323-19 2021-01-15T19:08:28.235 INFO:tasks.rados.rados.0.smithi001.stdout:115: finishing write tid 4 to smithi00140323-19 2021-01-15T19:08:28.235 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 30 (ObjNum 67 snap 8 seq_num 67) dirty exists 2021-01-15T19:08:28.236 INFO:tasks.rados.rados.0.smithi001.stdout:115: left oid 19 (ObjNum 67 snap 8 seq_num 67) 2021-01-15T19:08:28.236 INFO:tasks.rados.rados.0.smithi001.stdout:115: done (2 left) 2021-01-15T19:08:28.237 INFO:tasks.rados.rados.0.smithi001.stdout:118: rollback oid 10 current snap is 8 2021-01-15T19:08:28.237 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 10 to 7 2021-01-15T19:08:28.237 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 48 current snap is 8 2021-01-15T19:08:28.237 INFO:tasks.rados.rados.0.smithi001.stdout:119: seq_num 69 ranges {794624=761856,1556480=401408} 2021-01-15T19:08:28.244 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:28.245 INFO:tasks.rados.rados.0.smithi001.stdout:119: writing smithi00140323-48 from 794624 to 1556480 tid 1 2021-01-15T19:08:28.248 INFO:tasks.rados.rados.0.smithi001.stdout:119: writing smithi00140323-48 from 1556480 to 1957888 tid 2 2021-01-15T19:08:28.249 INFO:tasks.rados.rados.0.smithi001.stdout:120: snap_create 2021-01-15T19:08:28.249 INFO:tasks.rados.rados.0.smithi001.stdout:118: finishing rollback tid 1 to smithi00140323-10 2021-01-15T19:08:28.249 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 0 (ObjNum 1 snap 22026 seq_num 178803936) dirty dne 2021-01-15T19:08:28.249 INFO:tasks.rados.rados.0.smithi001.stdout:116: finishing write tid 1 to smithi00140323-26 2021-01-15T19:08:28.249 INFO:tasks.rados.rados.0.smithi001.stdout:116: finishing write tid 2 to smithi00140323-26 2021-01-15T19:08:28.250 INFO:tasks.rados.rados.0.smithi001.stdout:116: finishing write tid 3 to smithi00140323-26 2021-01-15T19:08:28.250 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 8 (ObjNum 20 snap 0 seq_num 20) dirty exists 2021-01-15T19:08:28.250 INFO:tasks.rados.rados.0.smithi001.stdout:116: finishing write tid 4 to smithi00140323-26 2021-01-15T19:08:28.250 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 25 (ObjNum 68 snap 8 seq_num 68) dirty exists 2021-01-15T19:08:28.250 INFO:tasks.rados.rados.0.smithi001.stdout:116: left oid 26 (ObjNum 68 snap 8 seq_num 68) 2021-01-15T19:08:28.253 INFO:tasks.rados.rados.0.smithi001.stdout:119: finishing write tid 1 to smithi00140323-48 2021-01-15T19:08:28.256 INFO:tasks.rados.rados.0.smithi001.stdout:119: finishing write tid 2 to smithi00140323-48 2021-01-15T19:08:28.256 INFO:tasks.rados.rados.0.smithi001.stdout:119: finishing write tid 3 to smithi00140323-48 2021-01-15T19:08:28.258 INFO:tasks.rados.rados.0.smithi001.stdout:119: finishing write tid 4 to smithi00140323-48 2021-01-15T19:08:28.258 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 10 (ObjNum 69 snap 8 seq_num 69) dirty exists 2021-01-15T19:08:28.258 INFO:tasks.rados.rados.0.smithi001.stdout:119: left oid 48 (ObjNum 69 snap 8 seq_num 69) 2021-01-15T19:08:28.258 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:28.256+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:28.267 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:28.267 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:28.268 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:28.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:28.289 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:28.310 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:28.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:28.361 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:28.359+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:28.442 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:28.443 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:28.444 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:28.452 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:28.492 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:28.490+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:28.561 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:28.558+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:28.618 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:28.619 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:28.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:28.619 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:28.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.620 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 26/21 ECSubWrite(tid=62, reqid=client.4418.0:170, at_version=26'4, trim_to=0'0, roll_forward_to=25'2))", 2021-01-15T19:08:28.620 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:28.620 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.359023479999999, 2021-01-15T19:08:28.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999, 2021-01-15T19:08:28.621 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.621 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.621 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:28.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.255965+0000", 2021-01-15T19:08:28.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4530000000000001e-05 2021-01-15T19:08:28.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256020+0000", 2021-01-15T19:08:28.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4555000000000001e-05 2021-01-15T19:08:28.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256044+0000", 2021-01-15T19:08:28.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9589000000000002e-05 2021-01-15T19:08:28.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256064+0000", 2021-01-15T19:08:28.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1316e-05 2021-01-15T19:08:28.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256075+0000", 2021-01-15T19:08:28.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3517999999999996e-05 2021-01-15T19:08:28.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256149+0000", 2021-01-15T19:08:28.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2123999999999999e-05 2021-01-15T19:08:28.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.256161+0000", 2021-01-15T19:08:28.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025493310000000002 2021-01-15T19:08:28.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258710+0000", 2021-01-15T19:08:28.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0933999999999997e-05 2021-01-15T19:08:28.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.258761+0000", 2021-01-15T19:08:28.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027958969999999999 2021-01-15T19:08:28.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.632 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.632 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:177 3.1s0 3:8f2f8304:::smithi00140323-26:head [create,setxattr TYVdm^TkaYX`fqU_WgdS[ (771) in=21b,setxattr f_Rg_SPVYXqfgb]QVYZgiYYTQrNnTm\\iqPj`S[oX (735) in=40b,setxattr ViSgWb (771) in=6b,setxattr gNWngcOqeYlbsc]qNpZ[\\Qh^sR]^gjjVVsmT\\t[T (15577) in=40b,setxattr \\oldYkYmX^d]eV[jYUcW_ganr\\^Ra\\Uol\\NXidYX (1471) in=40b,setxattr af_bh]^cNPWUcXPcYhaphRbX]`^aZT (771) in=30b,setxattr phlRVm^YeOWVcPfN\\raZjTahtlplZo (771) in=30b,setxattr sdfc]PTTrgf`eSfsms`r[fZtYdniafia^g`j^emY (841) in=40b,setxattr k]kV_ (771) in=5b,setxattr tQWZZa_aOpbaTq[_Og^WhcNcoUkPjnZP`fi[kdgg (1473) in=40b,setxattr SffWTkgf`VrioQ^nPna^r^a]WeijXRWWbSYS\\Saf (946) in=40b,setxattr fOPpkV]g^d_VWgT[Rhjdktrfnp]VY^einlU[[Nns (453) in=40b,setxattr gY`kggl\\bNjfaZg`ScVkkpU_oZQoXf_`SOk`XtTs (1998) in=40b,setxattr _QQjZRppX^cWk]_PkaRlpgfedp`W_oSQg\\_^kebP (1612) in=40b,setxattr ac^r`S\\US_\\rUm`p_XWtTa_phh`aZoXjU`grRdmq (8912) in=40b,setxattr PaRjqcagYWeje]OfYRrO^OV[sYk[OVeNhesY]Q]q (1507) in=40b,setxattr iUfUd_rQssfR`sV[dhnXW_keciQbe^j\\UUijdgUV (19016) in=40b,setxattr p\\XNRhPQrhsa_fbVakSeOm^e[qVt]rVe_VattOoT (1226) in=40b,setxattr fOpZofoOfPceYisPZkVhdSrSYsbRa_id_ZdoblbP (1682) in=40b,setxattr n (771) in=1b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:28.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:28.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.333443738, 2021-01-15T19:08:28.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494, 2021-01-15T19:08:28.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.634 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 177 2021-01-15T19:08:28.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.635 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:28.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281545+0000", 2021-01-15T19:08:28.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6885000000000005e-05 2021-01-15T19:08:28.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281642+0000", 2021-01-15T19:08:28.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:08:28.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281650+0000", 2021-01-15T19:08:28.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2979999999999999e-05 2021-01-15T19:08:28.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281663+0000", 2021-01-15T19:08:28.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1930000000000001e-06 2021-01-15T19:08:28.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281669+0000", 2021-01-15T19:08:28.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4880999999999999e-05 2021-01-15T19:08:28.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281724+0000", 2021-01-15T19:08:28.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946600000000001 2021-01-15T19:08:28.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.281853+0000", 2021-01-15T19:08:28.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000582403 2021-01-15T19:08:28.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.282435+0000", 2021-01-15T19:08:28.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013959510000000001 2021-01-15T19:08:28.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.283831+0000", 2021-01-15T19:08:28.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010843630000000001 2021-01-15T19:08:28.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284916+0000", 2021-01-15T19:08:28.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5765000000000005e-05 2021-01-15T19:08:28.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.284981+0000", 2021-01-15T19:08:28.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003436494 2021-01-15T19:08:28.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.648 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=71, reqid=client.4418.0:183, at_version=27'8, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:28.648 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:28.648 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.363292001, 2021-01-15T19:08:28.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003, 2021-01-15T19:08:28.648 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.649 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.649 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:28.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:28.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190392 2021-01-15T19:08:28.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251887+0000", 2021-01-15T19:08:28.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6526999999999999e-05 2021-01-15T19:08:28.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251914+0000", 2021-01-15T19:08:28.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8068999999999999e-05 2021-01-15T19:08:28.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251932+0000", 2021-01-15T19:08:28.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7269999999999996e-06 2021-01-15T19:08:28.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251940+0000", 2021-01-15T19:08:28.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012910800000000001 2021-01-15T19:08:28.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252070+0000", 2021-01-15T19:08:28.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2417e-05 2021-01-15T19:08:28.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252082+0000", 2021-01-15T19:08:28.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050434080000000001 2021-01-15T19:08:28.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257125+0000", 2021-01-15T19:08:28.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6032999999999998e-05 2021-01-15T19:08:28.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257171+0000", 2021-01-15T19:08:28.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003 2021-01-15T19:08:28.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.664 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=76, reqid=client.4418.0:184, at_version=27'9, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:28.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:28.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.360528815, 2021-01-15T19:08:28.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032503499999999999, 2021-01-15T19:08:28.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.665 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.665 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:28.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254460+0000", 2021-01-15T19:08:28.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012968499999999999 2021-01-15T19:08:28.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254590+0000", 2021-01-15T19:08:28.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5418999999999999e-05 2021-01-15T19:08:28.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254605+0000", 2021-01-15T19:08:28.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999996e-06 2021-01-15T19:08:28.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254615+0000", 2021-01-15T19:08:28.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7790000000000003e-06 2021-01-15T19:08:28.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254619+0000", 2021-01-15T19:08:28.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3176000000000001e-05 2021-01-15T19:08:28.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254663+0000", 2021-01-15T19:08:28.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1525000000000001e-05 2021-01-15T19:08:28.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254674+0000", 2021-01-15T19:08:28.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030008919999999998 2021-01-15T19:08:28.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257675+0000", 2021-01-15T19:08:28.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5274000000000001e-05 2021-01-15T19:08:28.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257710+0000", 2021-01-15T19:08:28.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032503499999999999 2021-01-15T19:08:28.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.676 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.676 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:28.677 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:28.677 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.359440641999999, 2021-01-15T19:08:28.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:28.677 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.677 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:28.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:28.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:28.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:28.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:28.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:28.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:28.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:28.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:28.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:28.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:28.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:28.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:28.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:28.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:28.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:28.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:28.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:28.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:28.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:28.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:28.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:28.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:28.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:28.691 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.691 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.691 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.692 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=79, reqid=client.4418.0:185, at_version=27'10, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:28.692 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:28.692 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.358245683, 2021-01-15T19:08:28.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999, 2021-01-15T19:08:28.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.693 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:28.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5377000000000001e-05 2021-01-15T19:08:28.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256768+0000", 2021-01-15T19:08:28.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3387000000000001e-05 2021-01-15T19:08:28.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256782+0000", 2021-01-15T19:08:28.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6764999999999998e-05 2021-01-15T19:08:28.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256819+0000", 2021-01-15T19:08:28.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:28.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256826+0000", 2021-01-15T19:08:28.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9475999999999998e-05 2021-01-15T19:08:28.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256886+0000", 2021-01-15T19:08:28.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1929e-05 2021-01-15T19:08:28.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256898+0000", 2021-01-15T19:08:28.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055282589999999998 2021-01-15T19:08:28.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262426+0000", 2021-01-15T19:08:28.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2537999999999997e-05 2021-01-15T19:08:28.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262479+0000", 2021-01-15T19:08:28.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999 2021-01-15T19:08:28.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.704 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.704 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:28.705 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:28.705 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.356714002, 2021-01-15T19:08:28.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:28.705 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.706 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.706 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:28.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.707 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:28.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:28.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:28.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:28.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:28.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:28.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:28.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:28.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:28.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:28.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:28.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:28.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:28.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:28.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:28.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:28.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:28.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:28.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:28.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:28.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:28.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:28.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:28.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:28.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:28.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.355789793, 2021-01-15T19:08:28.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:28.722 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.722 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.722 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.723 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:28.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.723 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:28.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:28.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:28.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:28.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:28.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:28.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:28.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:28.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:28.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:28.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:28.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:28.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:28.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:28.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:28.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:28.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:28.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:28.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:28.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:28.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:28.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:28.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:28.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.35556583, 2021-01-15T19:08:28.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:28.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:28.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:28.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.739 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:28.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:28.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:28.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:28.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:28.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:28.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:28.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:28.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:28.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:28.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:28.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:28.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:28.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:28.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:28.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:28.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:28.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:28.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:28.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:28.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.751 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=81, reqid=client.4418.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=27'10))", 2021-01-15T19:08:28.751 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:28.751 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.351989723000001, 2021-01-15T19:08:28.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002, 2021-01-15T19:08:28.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.752 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:28.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:28.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4539999999999999e-05 2021-01-15T19:08:28.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263024+0000", 2021-01-15T19:08:28.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9600000000000006e-06 2021-01-15T19:08:28.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263032+0000", 2021-01-15T19:08:28.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2021-01-15T19:08:28.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263041+0000", 2021-01-15T19:08:28.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3239999999999996e-06 2021-01-15T19:08:28.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263046+0000", 2021-01-15T19:08:28.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4546999999999999e-05 2021-01-15T19:08:28.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263090+0000", 2021-01-15T19:08:28.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0596e-05 2021-01-15T19:08:28.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263101+0000", 2021-01-15T19:08:28.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052094810000000002 2021-01-15T19:08:28.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268310+0000", 2021-01-15T19:08:28.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4725000000000003e-05 2021-01-15T19:08:28.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268355+0000", 2021-01-15T19:08:28.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002 2021-01-15T19:08:28.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.762 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.763 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:28.763 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:28.763 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.351703244999999, 2021-01-15T19:08:28.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999, 2021-01-15T19:08:28.764 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.764 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.764 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:28.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:28.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3807000000000001e-05 2021-01-15T19:08:28.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263309+0000", 2021-01-15T19:08:28.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:28.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263317+0000", 2021-01-15T19:08:28.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2021-01-15T19:08:28.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263327+0000", 2021-01-15T19:08:28.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9009999999999997e-06 2021-01-15T19:08:28.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263334+0000", 2021-01-15T19:08:28.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052278899999999998 2021-01-15T19:08:28.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268562+0000", 2021-01-15T19:08:28.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4100000000000001e-05 2021-01-15T19:08:28.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268576+0000", 2021-01-15T19:08:28.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999 2021-01-15T19:08:28.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.773 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=30, last_complete=27'11, committed=1, applied=1))", 2021-01-15T19:08:28.773 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:28.773 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.351032536, 2021-01-15T19:08:28.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048998940000000001, 2021-01-15T19:08:28.774 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.774 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.774 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263956+0000", 2021-01-15T19:08:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9382000000000001e-05 2021-01-15T19:08:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263986+0000", 2021-01-15T19:08:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.117e-06 2021-01-15T19:08:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263992+0000", 2021-01-15T19:08:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2400000000000007e-06 2021-01-15T19:08:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264000+0000", 2021-01-15T19:08:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2640000000000002e-06 2021-01-15T19:08:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264004+0000", 2021-01-15T19:08:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046074050000000002 2021-01-15T19:08:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268612+0000", 2021-01-15T19:08:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024448599999999998 2021-01-15T19:08:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268856+0000", 2021-01-15T19:08:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048998940000000001 2021-01-15T19:08:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3312067970000001, 2021-01-15T19:08:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:28.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:28.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3290275930000002, 2021-01-15T19:08:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:28.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3282732289999997, 2021-01-15T19:08:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3281530630000002, 2021-01-15T19:08:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:249 3.1s0 3:8f2f8304:::smithi00140323-26:7 [read 0~425984,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.300463+0000", 2021-01-15T19:08:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3145254209999999, 2021-01-15T19:08:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041641680000000002, 2021-01-15T19:08:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 249 2021-01-15T19:08:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300463+0000", 2021-01-15T19:08:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300463+0000", 2021-01-15T19:08:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9312e-05 2021-01-15T19:08:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300493+0000", 2021-01-15T19:08:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3279999999999996e-06 2021-01-15T19:08:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300498+0000", 2021-01-15T19:08:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3726999999999999e-05 2021-01-15T19:08:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300512+0000", 2021-01-15T19:08:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6409999999999999e-06 2021-01-15T19:08:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300517+0000", 2021-01-15T19:08:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7755e-05 2021-01-15T19:08:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300555+0000", 2021-01-15T19:08:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1619000000000006e-05 2021-01-15T19:08:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300637+0000", 2021-01-15T19:08:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039907859999999996 2021-01-15T19:08:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.304627+0000", 2021-01-15T19:08:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041641680000000002 2021-01-15T19:08:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:250 3.1s0 3:8f2f8304:::smithi00140323-26:7 [read 0~425984,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.300567+0000", 2021-01-15T19:08:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3144216909999997, 2021-01-15T19:08:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045113929999999998, 2021-01-15T19:08:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 250 2021-01-15T19:08:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300567+0000", 2021-01-15T19:08:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300567+0000", 2021-01-15T19:08:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6915999999999997e-05 2021-01-15T19:08:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300604+0000", 2021-01-15T19:08:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8520000000000003e-06 2021-01-15T19:08:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300609+0000", 2021-01-15T19:08:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1180000000000003e-06 2021-01-15T19:08:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300616+0000", 2021-01-15T19:08:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1159999999999999e-06 2021-01-15T19:08:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300619+0000", 2021-01-15T19:08:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028046400000000002 2021-01-15T19:08:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300899+0000", 2021-01-15T19:08:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102706 2021-01-15T19:08:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.301002+0000", 2021-01-15T19:08:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040762209999999997 2021-01-15T19:08:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.305078+0000", 2021-01-15T19:08:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045113929999999998 2021-01-15T19:08:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:251 3.1s0 3:8f2f8304:::smithi00140323-26:7 [read 0~425984,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.300643+0000", 2021-01-15T19:08:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.314345425, 2021-01-15T19:08:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049458740000000003, 2021-01-15T19:08:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 251 2021-01-15T19:08:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300643+0000", 2021-01-15T19:08:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300643+0000", 2021-01-15T19:08:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6999999999999999e-05 2021-01-15T19:08:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300670+0000", 2021-01-15T19:08:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1200000000000002e-06 2021-01-15T19:08:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300673+0000", 2021-01-15T19:08:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.241e-06 2021-01-15T19:08:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300680+0000", 2021-01-15T19:08:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.605e-06 2021-01-15T19:08:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300682+0000", 2021-01-15T19:08:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052756199999999997 2021-01-15T19:08:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.301210+0000", 2021-01-15T19:08:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9424000000000004e-05 2021-01-15T19:08:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.301289+0000", 2021-01-15T19:08:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042999220000000003 2021-01-15T19:08:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.305589+0000", 2021-01-15T19:08:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049458740000000003 2021-01-15T19:08:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.1s0 36/21 ECSubRead(tid=49, to_read={3:8f2f8304:::smithi00140323-26:head=0,212992,0}, subchunks={3:8f2f8304:::smithi00140323-26:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:08:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.300833+0000", 2021-01-15T19:08:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3141559809999999, 2021-01-15T19:08:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028423139999999999, 2021-01-15T19:08:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300833+0000", 2021-01-15T19:08:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300833+0000", 2021-01-15T19:08:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300833+0000", 2021-01-15T19:08:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300833+0000", 2021-01-15T19:08:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000147971 2021-01-15T19:08:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300981+0000", 2021-01-15T19:08:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9059999999999996e-06 2021-01-15T19:08:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300988+0000", 2021-01-15T19:08:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051068900000000004 2021-01-15T19:08:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.301498+0000", 2021-01-15T19:08:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021767480000000001 2021-01-15T19:08:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.303675+0000", 2021-01-15T19:08:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028423139999999999 2021-01-15T19:08:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:28.890 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:28.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:28.893 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:28.891+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:28.992 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:28.990+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:29.058 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:29.059 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:29.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:29.093 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:29.092+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:29.179 INFO:tasks.rados.rados.0.smithi001.stdout:116: done (4 left) 2021-01-15T19:08:29.180 INFO:tasks.rados.rados.0.smithi001.stdout:117: done (3 left) 2021-01-15T19:08:29.180 INFO:tasks.rados.rados.0.smithi001.stdout:118: done (2 left) 2021-01-15T19:08:29.180 INFO:tasks.rados.rados.0.smithi001.stdout:119: done (1 left) 2021-01-15T19:08:29.181 INFO:tasks.rados.rados.0.smithi001.stdout:120: done (0 left) 2021-01-15T19:08:29.181 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 48 current snap is 9 2021-01-15T19:08:29.181 INFO:tasks.rados.rados.0.smithi001.stdout:121: seq_num 70 ranges {1957888=778240,2736128=401408} 2021-01-15T19:08:29.194 INFO:tasks.rados.rados.0.smithi001.stdout:121: writing smithi00140323-48 from 1957888 to 2736128 tid 1 2021-01-15T19:08:29.194 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:29.191+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:29.196 INFO:tasks.rados.rados.0.smithi001.stdout:121: writing smithi00140323-48 from 2736128 to 3137536 tid 2 2021-01-15T19:08:29.196 INFO:tasks.rados.rados.0.smithi001.stdout:122: read oid 35 snap -1 2021-01-15T19:08:29.197 INFO:tasks.rados.rados.0.smithi001.stdout:122: expect (ObjNum 34 snap 0 seq_num 34) 2021-01-15T19:08:29.197 INFO:tasks.rados.rados.0.smithi001.stdout:123: copy_from oid 24 from oid 38 current snap is 9 2021-01-15T19:08:29.197 INFO:tasks.rados.rados.0.smithi001.stdout:124: rollback oid 9 current snap is 9 2021-01-15T19:08:29.197 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 9 to 7 2021-01-15T19:08:29.197 INFO:tasks.rados.rados.0.smithi001.stdout:125: delete oid 14 current snap is 9 2021-01-15T19:08:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:125: done (4 left) 2021-01-15T19:08:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 32 current snap is 9 2021-01-15T19:08:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:126: seq_num 72 ranges {729088=409600,1138688=524288,1662976=278528} 2021-01-15T19:08:29.210 INFO:tasks.rados.rados.0.smithi001.stdout:126: writing smithi00140323-32 from 729088 to 1138688 tid 1 2021-01-15T19:08:29.213 INFO:tasks.rados.rados.0.smithi001.stdout:126: writing smithi00140323-32 from 1138688 to 1662976 tid 2 2021-01-15T19:08:29.216 INFO:tasks.rados.rados.0.smithi001.stdout:126: writing smithi00140323-32 from 1662976 to 1941504 tid 3 2021-01-15T19:08:29.216 INFO:tasks.rados.rados.0.smithi001.stdout:127: read oid 10 snap 8 2021-01-15T19:08:29.217 INFO:tasks.rados.rados.0.smithi001.stdout:127: expect deleted 2021-01-15T19:08:29.218 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 47 current snap is 9 2021-01-15T19:08:29.218 INFO:tasks.rados.rados.0.smithi001.stdout:124: finishing rollback tid 0 to smithi00140323-9 2021-01-15T19:08:29.219 INFO:tasks.rados.rados.0.smithi001.stdout:124: finishing rollback tid 1 to smithi00140323-9 2021-01-15T19:08:29.220 INFO:tasks.rados.rados.0.smithi001.stdout:128: seq_num 73 ranges {0=425984,425984=409600,835584=393216} 2021-01-15T19:08:29.220 INFO:tasks.rados.rados.0.smithi001.stdout:128: writing smithi00140323-47 from 0 to 425984 tid 1 2021-01-15T19:08:29.221 INFO:tasks.rados.rados.0.smithi001.stdout:128: writing smithi00140323-47 from 425984 to 835584 tid 2 2021-01-15T19:08:29.223 INFO:tasks.rados.rados.0.smithi001.stdout:128: writing smithi00140323-47 from 835584 to 1228800 tid 3 2021-01-15T19:08:29.223 INFO:tasks.rados.rados.0.smithi001.stdout:129: snap_create 2021-01-15T19:08:29.223 INFO:tasks.rados.rados.0.smithi001.stdout:124: finishing rollback tid 2 to smithi00140323-9 2021-01-15T19:08:29.224 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 9 v 21 (ObjNum 8 snap 0 seq_num 8) dirty exists 2021-01-15T19:08:29.224 INFO:tasks.rados.rados.0.smithi001.stdout:121: finishing write tid 1 to smithi00140323-48 2021-01-15T19:08:29.224 INFO:tasks.rados.rados.0.smithi001.stdout:121: finishing write tid 2 to smithi00140323-48 2021-01-15T19:08:29.225 INFO:tasks.rados.rados.0.smithi001.stdout:122: expect (ObjNum 34 snap 0 seq_num 34) 2021-01-15T19:08:29.238 INFO:tasks.rados.rados.0.smithi001.stdout:121: finishing write tid 3 to smithi00140323-48 2021-01-15T19:08:29.238 INFO:tasks.rados.rados.0.smithi001.stdout:121: finishing write tid 4 to smithi00140323-48 2021-01-15T19:08:29.239 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 14 (ObjNum 70 snap 9 seq_num 70) dirty exists 2021-01-15T19:08:29.239 INFO:tasks.rados.rados.0.smithi001.stdout:121: left oid 48 (ObjNum 70 snap 9 seq_num 70) 2021-01-15T19:08:29.239 INFO:tasks.rados.rados.0.smithi001.stdout:123: finishing copy_from to smithi00140323-24 2021-01-15T19:08:29.239 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 24 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2021-01-15T19:08:29.239 INFO:tasks.rados.rados.0.smithi001.stdout:123: finishing copy_from racing read to smithi00140323-24 2021-01-15T19:08:29.240 INFO:tasks.rados.rados.0.smithi001.stdout:126: finishing write tid 1 to smithi00140323-32 2021-01-15T19:08:29.240 INFO:tasks.rados.rados.0.smithi001.stdout:126: finishing write tid 2 to smithi00140323-32 2021-01-15T19:08:29.240 INFO:tasks.rados.rados.0.smithi001.stdout:126: finishing write tid 3 to smithi00140323-32 2021-01-15T19:08:29.240 INFO:tasks.rados.rados.0.smithi001.stdout:126: finishing write tid 4 to smithi00140323-32 2021-01-15T19:08:29.241 INFO:tasks.rados.rados.0.smithi001.stdout:126: finishing write tid 5 to smithi00140323-32 2021-01-15T19:08:29.241 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 26 (ObjNum 72 snap 9 seq_num 72) dirty exists 2021-01-15T19:08:29.241 INFO:tasks.rados.rados.0.smithi001.stdout:126: left oid 32 (ObjNum 72 snap 9 seq_num 72) 2021-01-15T19:08:29.241 INFO:tasks.rados.rados.0.smithi001.stdout:128: finishing write tid 1 to smithi00140323-47 2021-01-15T19:08:29.241 INFO:tasks.rados.rados.0.smithi001.stdout:128: finishing write tid 2 to smithi00140323-47 2021-01-15T19:08:29.242 INFO:tasks.rados.rados.0.smithi001.stdout:128: finishing write tid 3 to smithi00140323-47 2021-01-15T19:08:29.242 INFO:tasks.rados.rados.0.smithi001.stdout:128: finishing write tid 4 to smithi00140323-47 2021-01-15T19:08:29.242 INFO:tasks.rados.rados.0.smithi001.stdout:128: finishing write tid 5 to smithi00140323-47 2021-01-15T19:08:29.242 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 26 (ObjNum 73 snap 9 seq_num 73) dirty exists 2021-01-15T19:08:29.242 INFO:tasks.rados.rados.0.smithi001.stdout:128: left oid 47 (ObjNum 73 snap 9 seq_num 73) 2021-01-15T19:08:29.246 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:29.247 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:29.247 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:29.248 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:29.248 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:29.256 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:29.297 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:29.295+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:29.398 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:29.392+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:29.423 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:29.423 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:29.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:29.424 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:29.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 26/21 ECSubWrite(tid=34, reqid=client.4418.0:156, at_version=26'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:29.425 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:29.425 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.176835457999999, 2021-01-15T19:08:29.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002, 2021-01-15T19:08:29.425 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.426 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.426 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:29.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243362+0000", 2021-01-15T19:08:29.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166446 2021-01-15T19:08:29.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243528+0000", 2021-01-15T19:08:29.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5724e-05 2021-01-15T19:08:29.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243554+0000", 2021-01-15T19:08:29.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3528e-05 2021-01-15T19:08:29.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243567+0000", 2021-01-15T19:08:29.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4610000000000008e-06 2021-01-15T19:08:29.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243576+0000", 2021-01-15T19:08:29.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1644999999999999e-05 2021-01-15T19:08:29.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243618+0000", 2021-01-15T19:08:29.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2930000000000001e-06 2021-01-15T19:08:29.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.243626+0000", 2021-01-15T19:08:29.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020585310000000002 2021-01-15T19:08:29.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245684+0000", 2021-01-15T19:08:29.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7432999999999997e-05 2021-01-15T19:08:29.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:14.245742+0000", 2021-01-15T19:08:29.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023800610000000002 2021-01-15T19:08:29.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=74, reqid=client.4418.0:190, at_version=27'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:29.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:29.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.168364318, 2021-01-15T19:08:29.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002, 2021-01-15T19:08:29.439 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.439 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.439 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:29.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:29.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8488000000000001e-05 2021-01-15T19:08:29.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251871+0000", 2021-01-15T19:08:29.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0378e-05 2021-01-15T19:08:29.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251902+0000", 2021-01-15T19:08:29.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3121e-05 2021-01-15T19:08:29.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251915+0000", 2021-01-15T19:08:29.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3499999999999996e-06 2021-01-15T19:08:29.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251920+0000", 2021-01-15T19:08:29.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108286 2021-01-15T19:08:29.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252028+0000", 2021-01-15T19:08:29.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1652e-05 2021-01-15T19:08:29.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252040+0000", 2021-01-15T19:08:29.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002924991 2021-01-15T19:08:29.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254965+0000", 2021-01-15T19:08:29.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4570000000000001e-05 2021-01-15T19:08:29.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255020+0000", 2021-01-15T19:08:29.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002 2021-01-15T19:08:29.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=75, reqid=client.4418.0:191, at_version=27'13, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:29.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:29.452 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.166857189, 2021-01-15T19:08:29.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022390299999999999, 2021-01-15T19:08:29.452 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.452 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.452 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:29.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253340+0000", 2021-01-15T19:08:29.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0649999999999998e-05 2021-01-15T19:08:29.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253391+0000", 2021-01-15T19:08:29.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0658000000000001e-05 2021-01-15T19:08:29.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253411+0000", 2021-01-15T19:08:29.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8773000000000001e-05 2021-01-15T19:08:29.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253430+0000", 2021-01-15T19:08:29.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4130000000000006e-06 2021-01-15T19:08:29.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253438+0000", 2021-01-15T19:08:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4926000000000003e-05 2021-01-15T19:08:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253483+0000", 2021-01-15T19:08:29.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3898e-05 2021-01-15T19:08:29.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.253497+0000", 2021-01-15T19:08:29.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020547130000000001 2021-01-15T19:08:29.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255552+0000", 2021-01-15T19:08:29.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6999e-05 2021-01-15T19:08:29.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255579+0000", 2021-01-15T19:08:29.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022390299999999999 2021-01-15T19:08:29.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.463 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.464 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=45, reqid=client.4418.0:204, at_version=28'16, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:29.464 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:29.465 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.164749931999999, 2021-01-15T19:08:29.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002, 2021-01-15T19:08:29.465 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.465 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.466 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:29.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:29.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1822999999999997e-05 2021-01-15T19:08:29.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255479+0000", 2021-01-15T19:08:29.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7513999999999999e-05 2021-01-15T19:08:29.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255497+0000", 2021-01-15T19:08:29.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3723000000000001e-05 2021-01-15T19:08:29.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255510+0000", 2021-01-15T19:08:29.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0780000000000003e-06 2021-01-15T19:08:29.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255515+0000", 2021-01-15T19:08:29.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2679999999999998e-05 2021-01-15T19:08:29.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255558+0000", 2021-01-15T19:08:29.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.695e-06 2021-01-15T19:08:29.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255566+0000", 2021-01-15T19:08:29.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028394660000000001 2021-01-15T19:08:29.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258405+0000", 2021-01-15T19:08:29.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9791999999999999e-05 2021-01-15T19:08:29.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258435+0000", 2021-01-15T19:08:29.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002 2021-01-15T19:08:29.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.478 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.479 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 36/21 ECSubWrite(tid=30, reqid=client.4418.0:244, at_version=36'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:29.479 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:29.479 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1273661700000002, 2021-01-15T19:08:29.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034199159999999998, 2021-01-15T19:08:29.480 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.480 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:29.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:29.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5223999999999999e-05 2021-01-15T19:08:29.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292876+0000", 2021-01-15T19:08:29.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.614e-05 2021-01-15T19:08:29.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292902+0000", 2021-01-15T19:08:29.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2162e-05 2021-01-15T19:08:29.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292915+0000", 2021-01-15T19:08:29.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2430000000000005e-06 2021-01-15T19:08:29.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292923+0000", 2021-01-15T19:08:29.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4569e-05 2021-01-15T19:08:29.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292967+0000", 2021-01-15T19:08:29.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2889999999999998e-06 2021-01-15T19:08:29.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292976+0000", 2021-01-15T19:08:29.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032267860000000002 2021-01-15T19:08:29.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296202+0000", 2021-01-15T19:08:29.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8503e-05 2021-01-15T19:08:29.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296251+0000", 2021-01-15T19:08:29.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034199159999999998 2021-01-15T19:08:29.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/21 ECSubWrite(tid=71, reqid=client.4418.0:252, at_version=36'23, trim_to=0'0, roll_forward_to=34'21))", 2021-01-15T19:08:29.497 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:29.497 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1086642710000003, 2021-01-15T19:08:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003352748, 2021-01-15T19:08:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:29.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:29.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047739199999999999 2021-01-15T19:08:29.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312010+0000", 2021-01-15T19:08:29.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1661000000000001e-05 2021-01-15T19:08:29.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312032+0000", 2021-01-15T19:08:29.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1812e-05 2021-01-15T19:08:29.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312044+0000", 2021-01-15T19:08:29.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.038e-06 2021-01-15T19:08:29.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312052+0000", 2021-01-15T19:08:29.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3818999999999993e-05 2021-01-15T19:08:29.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312116+0000", 2021-01-15T19:08:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4571e-05 2021-01-15T19:08:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312130+0000", 2021-01-15T19:08:29.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002722431 2021-01-15T19:08:29.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314853+0000", 2021-01-15T19:08:29.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3024000000000001e-05 2021-01-15T19:08:29.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314886+0000", 2021-01-15T19:08:29.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003352748 2021-01-15T19:08:29.508 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.508 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.509 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.509 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/21 ECSubWrite(tid=32, reqid=osd.5.0:32, at_version=37'14, trim_to=0'0, roll_forward_to=36'12))", 2021-01-15T19:08:29.509 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:25.282532+0000", 2021-01-15T19:08:29.510 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1376649580000002, 2021-01-15T19:08:29.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029664169999999998, 2021-01-15T19:08:29.510 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.510 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.510 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282532+0000", 2021-01-15T19:08:29.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282532+0000", 2021-01-15T19:08:29.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6085999999999998e-05 2021-01-15T19:08:29.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282588+0000", 2021-01-15T19:08:29.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0889999999999997e-05 2021-01-15T19:08:29.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282619+0000", 2021-01-15T19:08:29.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7946999999999999e-05 2021-01-15T19:08:29.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282637+0000", 2021-01-15T19:08:29.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4449999999999996e-06 2021-01-15T19:08:29.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282646+0000", 2021-01-15T19:08:29.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2564000000000001e-05 2021-01-15T19:08:29.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282688+0000", 2021-01-15T19:08:29.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.85e-06 2021-01-15T19:08:29.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282697+0000", 2021-01-15T19:08:29.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027569029999999998 2021-01-15T19:08:29.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.285454+0000", 2021-01-15T19:08:29.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4731999999999998e-05 2021-01-15T19:08:29.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.285499+0000", 2021-01-15T19:08:29.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029664169999999998 2021-01-15T19:08:29.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/21 ECSubWrite(tid=33, reqid=osd.5.0:32, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2021-01-15T19:08:29.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:25.286039+0000", 2021-01-15T19:08:29.522 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1341577660000004, 2021-01-15T19:08:29.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024049200000000001, 2021-01-15T19:08:29.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.523 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286039+0000", 2021-01-15T19:08:29.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286039+0000", 2021-01-15T19:08:29.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9084e-05 2021-01-15T19:08:29.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286088+0000", 2021-01-15T19:08:29.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6044000000000001e-05 2021-01-15T19:08:29.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286105+0000", 2021-01-15T19:08:29.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3998999999999999e-05 2021-01-15T19:08:29.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286119+0000", 2021-01-15T19:08:29.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4630000000000001e-06 2021-01-15T19:08:29.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286127+0000", 2021-01-15T19:08:29.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2855000000000002e-05 2021-01-15T19:08:29.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286170+0000", 2021-01-15T19:08:29.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9319999999999996e-06 2021-01-15T19:08:29.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.286178+0000", 2021-01-15T19:08:29.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021825949999999998 2021-01-15T19:08:29.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.288360+0000", 2021-01-15T19:08:29.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3948000000000003e-05 2021-01-15T19:08:29.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.288444+0000", 2021-01-15T19:08:29.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024049200000000001 2021-01-15T19:08:29.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=84, reqid=client.4418.0:311, at_version=41'12, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:29.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:29.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.223936685, 2021-01-15T19:08:29.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049070149999999998, 2021-01-15T19:08:29.535 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.536 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:29.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:29.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065051599999999996 2021-01-15T19:08:29.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196911+0000", 2021-01-15T19:08:29.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2234000000000001e-05 2021-01-15T19:08:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196933+0000", 2021-01-15T19:08:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4019000000000001e-05 2021-01-15T19:08:29.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196947+0000", 2021-01-15T19:08:29.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1910000000000003e-06 2021-01-15T19:08:29.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196954+0000", 2021-01-15T19:08:29.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7198999999999999e-05 2021-01-15T19:08:29.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197012+0000", 2021-01-15T19:08:29.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5761999999999999e-05 2021-01-15T19:08:29.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197027+0000", 2021-01-15T19:08:29.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004104697 2021-01-15T19:08:29.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.201132+0000", 2021-01-15T19:08:29.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5397e-05 2021-01-15T19:08:29.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.201167+0000", 2021-01-15T19:08:29.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049070149999999998 2021-01-15T19:08:29.546 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.546 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.546 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.547 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=86, reqid=client.4418.0:312, at_version=41'13, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:29.547 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.202431+0000", 2021-01-15T19:08:29.547 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.21776574700000001, 2021-01-15T19:08:29.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025230610000000001, 2021-01-15T19:08:29.548 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.548 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.548 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202431+0000", 2021-01-15T19:08:29.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202431+0000", 2021-01-15T19:08:29.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000221342 2021-01-15T19:08:29.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202653+0000", 2021-01-15T19:08:29.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1331999999999998e-05 2021-01-15T19:08:29.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202674+0000", 2021-01-15T19:08:29.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8206000000000002e-05 2021-01-15T19:08:29.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202692+0000", 2021-01-15T19:08:29.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8369999999999999e-06 2021-01-15T19:08:29.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202699+0000", 2021-01-15T19:08:29.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2725000000000002e-05 2021-01-15T19:08:29.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202742+0000", 2021-01-15T19:08:29.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1755e-05 2021-01-15T19:08:29.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202754+0000", 2021-01-15T19:08:29.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021553089999999998 2021-01-15T19:08:29.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204909+0000", 2021-01-15T19:08:29.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5555e-05 2021-01-15T19:08:29.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204954+0000", 2021-01-15T19:08:29.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025230610000000001 2021-01-15T19:08:29.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.559 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=87, reqid=client.4418.0:313, at_version=41'14, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:29.559 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:29.560 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.21558635700000001, 2021-01-15T19:08:29.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998, 2021-01-15T19:08:29.560 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.560 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.560 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:29.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:29.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0902000000000002e-05 2021-01-15T19:08:29.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204662+0000", 2021-01-15T19:08:29.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8193e-05 2021-01-15T19:08:29.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204680+0000", 2021-01-15T19:08:29.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958e-05 2021-01-15T19:08:29.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204698+0000", 2021-01-15T19:08:29.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2021-01-15T19:08:29.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204706+0000", 2021-01-15T19:08:29.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2994000000000002e-05 2021-01-15T19:08:29.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204749+0000", 2021-01-15T19:08:29.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491999999999999e-05 2021-01-15T19:08:29.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204760+0000", 2021-01-15T19:08:29.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056169770000000004 2021-01-15T19:08:29.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210377+0000", 2021-01-15T19:08:29.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8433999999999997e-05 2021-01-15T19:08:29.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210425+0000", 2021-01-15T19:08:29.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998 2021-01-15T19:08:29.571 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.571 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.571 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.572 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=114, reqid=client.4418.0:322, at_version=0'0, trim_to=0'0, roll_forward_to=41'21))", 2021-01-15T19:08:29.572 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.208095+0000", 2021-01-15T19:08:29.572 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.212101756, 2021-01-15T19:08:29.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024870550000000002, 2021-01-15T19:08:29.572 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.572 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208095+0000", 2021-01-15T19:08:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208095+0000", 2021-01-15T19:08:29.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8841999999999998e-05 2021-01-15T19:08:29.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208144+0000", 2021-01-15T19:08:29.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9432999999999998e-05 2021-01-15T19:08:29.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208164+0000", 2021-01-15T19:08:29.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7906000000000001e-05 2021-01-15T19:08:29.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208182+0000", 2021-01-15T19:08:29.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8140000000000001e-06 2021-01-15T19:08:29.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208189+0000", 2021-01-15T19:08:29.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5748000000000002e-05 2021-01-15T19:08:29.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208245+0000", 2021-01-15T19:08:29.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7170999999999999e-05 2021-01-15T19:08:29.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208262+0000", 2021-01-15T19:08:29.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022647689999999998 2021-01-15T19:08:29.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210527+0000", 2021-01-15T19:08:29.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5371999999999998e-05 2021-01-15T19:08:29.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210582+0000", 2021-01-15T19:08:29.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024870550000000002 2021-01-15T19:08:29.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=91, reqid=client.4418.0:313, at_version=0'0, trim_to=0'0, roll_forward_to=41'14))", 2021-01-15T19:08:29.584 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.211528+0000", 2021-01-15T19:08:29.584 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20866886700000001, 2021-01-15T19:08:29.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003814684, 2021-01-15T19:08:29.584 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.585 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.585 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211528+0000", 2021-01-15T19:08:29.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211528+0000", 2021-01-15T19:08:29.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3132000000000005e-05 2021-01-15T19:08:29.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211591+0000", 2021-01-15T19:08:29.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6620999999999999e-05 2021-01-15T19:08:29.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211608+0000", 2021-01-15T19:08:29.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8450000000000001e-05 2021-01-15T19:08:29.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211627+0000", 2021-01-15T19:08:29.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1795e-05 2021-01-15T19:08:29.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211638+0000", 2021-01-15T19:08:29.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3019000000000002e-05 2021-01-15T19:08:29.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211691+0000", 2021-01-15T19:08:29.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3823e-05 2021-01-15T19:08:29.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211705+0000", 2021-01-15T19:08:29.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003581135 2021-01-15T19:08:29.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.215286+0000", 2021-01-15T19:08:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6709000000000002e-05 2021-01-15T19:08:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.215343+0000", 2021-01-15T19:08:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003814684 2021-01-15T19:08:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=115, reqid=client.4418.0:324, at_version=41'23, trim_to=0'0, roll_forward_to=41'21))", 2021-01-15T19:08:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.207966498, 2021-01-15T19:08:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00487003, 2021-01-15T19:08:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3484000000000002e-05 2021-01-15T19:08:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212314+0000", 2021-01-15T19:08:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7509000000000001e-05 2021-01-15T19:08:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212332+0000", 2021-01-15T19:08:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1646e-05 2021-01-15T19:08:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212343+0000", 2021-01-15T19:08:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:08:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212349+0000", 2021-01-15T19:08:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3396999999999997e-05 2021-01-15T19:08:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212402+0000", 2021-01-15T19:08:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3617e-05 2021-01-15T19:08:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212416+0000", 2021-01-15T19:08:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046101270000000003 2021-01-15T19:08:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217026+0000", 2021-01-15T19:08:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4484e-05 2021-01-15T19:08:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217101+0000", 2021-01-15T19:08:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00487003 2021-01-15T19:08:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=117, reqid=client.4418.0:326, at_version=41'25, trim_to=0'0, roll_forward_to=41'21))", 2021-01-15T19:08:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217776+0000", 2021-01-15T19:08:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20242086500000001, 2021-01-15T19:08:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033786580000000001, 2021-01-15T19:08:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217776+0000", 2021-01-15T19:08:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217776+0000", 2021-01-15T19:08:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014095899999999999 2021-01-15T19:08:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217917+0000", 2021-01-15T19:08:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2052000000000002e-05 2021-01-15T19:08:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217939+0000", 2021-01-15T19:08:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2021-01-15T19:08:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217958+0000", 2021-01-15T19:08:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7789999999999998e-06 2021-01-15T19:08:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217964+0000", 2021-01-15T19:08:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6045999999999999e-05 2021-01-15T19:08:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218011+0000", 2021-01-15T19:08:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2300000000000001e-05 2021-01-15T19:08:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218023+0000", 2021-01-15T19:08:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031022229999999999 2021-01-15T19:08:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221125+0000", 2021-01-15T19:08:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9929000000000001e-05 2021-01-15T19:08:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221155+0000", 2021-01-15T19:08:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033786580000000001 2021-01-15T19:08:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=42, reqid=client.4418.0:332, at_version=41'23, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.221646+0000", 2021-01-15T19:08:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.198551638, 2021-01-15T19:08:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031358509999999998, 2021-01-15T19:08:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221646+0000", 2021-01-15T19:08:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221646+0000", 2021-01-15T19:08:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010761500000000001 2021-01-15T19:08:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221753+0000", 2021-01-15T19:08:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4093e-05 2021-01-15T19:08:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221767+0000", 2021-01-15T19:08:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7309999999999999e-06 2021-01-15T19:08:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221776+0000", 2021-01-15T19:08:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3340000000000003e-06 2021-01-15T19:08:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221780+0000", 2021-01-15T19:08:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4490000000000003e-05 2021-01-15T19:08:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221825+0000", 2021-01-15T19:08:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2089999999999999e-06 2021-01-15T19:08:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221832+0000", 2021-01-15T19:08:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002886422 2021-01-15T19:08:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224718+0000", 2021-01-15T19:08:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2957e-05 2021-01-15T19:08:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224781+0000", 2021-01-15T19:08:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031358509999999998 2021-01-15T19:08:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=119, reqid=client.4418.0:327, at_version=0'0, trim_to=0'0, roll_forward_to=41'26))", 2021-01-15T19:08:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222670+0000", 2021-01-15T19:08:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19752676999999999, 2021-01-15T19:08:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039183689999999997, 2021-01-15T19:08:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222670+0000", 2021-01-15T19:08:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222670+0000", 2021-01-15T19:08:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4183e-05 2021-01-15T19:08:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222715+0000", 2021-01-15T19:08:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3796e-05 2021-01-15T19:08:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222728+0000", 2021-01-15T19:08:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7295999999999998e-05 2021-01-15T19:08:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222746+0000", 2021-01-15T19:08:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000002e-06 2021-01-15T19:08:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222752+0000", 2021-01-15T19:08:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4154999999999997e-05 2021-01-15T19:08:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222826+0000", 2021-01-15T19:08:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3285000000000001e-05 2021-01-15T19:08:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222839+0000", 2021-01-15T19:08:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037127660000000002 2021-01-15T19:08:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226552+0000", 2021-01-15T19:08:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6730000000000002e-05 2021-01-15T19:08:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226589+0000", 2021-01-15T19:08:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039183689999999997 2021-01-15T19:08:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.196597932, 2021-01-15T19:08:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=44, reqid=client.4418.0:334, at_version=41'25, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19490371500000001, 2021-01-15T19:08:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001, 2021-01-15T19:08:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9133000000000004e-05 2021-01-15T19:08:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225363+0000", 2021-01-15T19:08:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2021-01-15T19:08:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225376+0000", 2021-01-15T19:08:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0949e-05 2021-01-15T19:08:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225387+0000", 2021-01-15T19:08:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7710000000000001e-06 2021-01-15T19:08:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225394+0000", 2021-01-15T19:08:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5732999999999997e-05 2021-01-15T19:08:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225440+0000", 2021-01-15T19:08:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5984000000000001e-05 2021-01-15T19:08:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225456+0000", 2021-01-15T19:08:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059787429999999999 2021-01-15T19:08:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231434+0000", 2021-01-15T19:08:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2653000000000001e-05 2021-01-15T19:08:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231457+0000", 2021-01-15T19:08:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001 2021-01-15T19:08:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=45, reqid=client.4418.0:335, at_version=41'26, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19321698900000001, 2021-01-15T19:08:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047473949999999997, 2021-01-15T19:08:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0362000000000003e-05 2021-01-15T19:08:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227031+0000", 2021-01-15T19:08:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9890000000000001e-05 2021-01-15T19:08:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227050+0000", 2021-01-15T19:08:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7663e-05 2021-01-15T19:08:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227068+0000", 2021-01-15T19:08:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4250000000000003e-06 2021-01-15T19:08:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227075+0000", 2021-01-15T19:08:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8297999999999997e-05 2021-01-15T19:08:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227123+0000", 2021-01-15T19:08:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0387000000000001e-05 2021-01-15T19:08:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227133+0000", 2021-01-15T19:08:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004554598 2021-01-15T19:08:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231688+0000", 2021-01-15T19:08:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9771999999999998e-05 2021-01-15T19:08:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231728+0000", 2021-01-15T19:08:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047473949999999997 2021-01-15T19:08:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:29.682 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:29.683 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:29.681+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:29.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:29.784 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:29.782+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:29.828 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:29.829 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:29.830 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:29.830 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:29.839 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:29.922 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:29.921+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:29.986 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:29.984+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:30.010 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:30.011 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:30.011 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:30.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:30.104 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:30.103+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:30.180 INFO:tasks.rados.rados.0.smithi001.stdout:121: done (7 left) 2021-01-15T19:08:30.180 INFO:tasks.rados.rados.0.smithi001.stdout:122: done (6 left) 2021-01-15T19:08:30.180 INFO:tasks.rados.rados.0.smithi001.stdout:123: done (5 left) 2021-01-15T19:08:30.181 INFO:tasks.rados.rados.0.smithi001.stdout:124: done (4 left) 2021-01-15T19:08:30.181 INFO:tasks.rados.rados.0.smithi001.stdout:126: done (3 left) 2021-01-15T19:08:30.181 INFO:tasks.rados.rados.0.smithi001.stdout:127: done (2 left) 2021-01-15T19:08:30.181 INFO:tasks.rados.rados.0.smithi001.stdout:128: done (1 left) 2021-01-15T19:08:30.181 INFO:tasks.rados.rados.0.smithi001.stdout:129: done (0 left) 2021-01-15T19:08:30.181 INFO:tasks.rados.rados.0.smithi001.stdout:130: read oid 44 snap -1 2021-01-15T19:08:30.182 INFO:tasks.rados.rados.0.smithi001.stdout:130: expect (ObjNum 43 snap 0 seq_num 43) 2021-01-15T19:08:30.182 INFO:tasks.rados.rados.0.smithi001.stdout:131: snap_remove snap 8 2021-01-15T19:08:30.188 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:30.187+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:30.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:08:30.195 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:30.195 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:30.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:30.196 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:08:30.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.196 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:154 3.fs0 3:f27a3046:::smithi00140323-1:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:30.196 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:30.197 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.955394483999999, 2021-01-15T19:08:30.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002, 2021-01-15T19:08:30.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.198 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 154 2021-01-15T19:08:30.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.198 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:30.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:30.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4352999999999999e-05 2021-01-15T19:08:30.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237631+0000", 2021-01-15T19:08:30.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3304e-05 2021-01-15T19:08:30.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237644+0000", 2021-01-15T19:08:30.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1736e-05 2021-01-15T19:08:30.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237666+0000", 2021-01-15T19:08:30.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0607e-05 2021-01-15T19:08:30.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237676+0000", 2021-01-15T19:08:30.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3862e-05 2021-01-15T19:08:30.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237730+0000", 2021-01-15T19:08:30.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1849999999999998e-06 2021-01-15T19:08:30.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:30.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237737+0000", 2021-01-15T19:08:30.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014276250000000001 2021-01-15T19:08:30.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239165+0000", 2021-01-15T19:08:30.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7447000000000007e-05 2021-01-15T19:08:30.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239232+0000", 2021-01-15T19:08:30.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061349480000000003 2021-01-15T19:08:30.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245367+0000", 2021-01-15T19:08:30.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095382000000000002 2021-01-15T19:08:30.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254906+0000", 2021-01-15T19:08:30.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113465 2021-01-15T19:08:30.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255019+0000", 2021-01-15T19:08:30.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000454659 2021-01-15T19:08:30.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255474+0000", 2021-01-15T19:08:30.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002 2021-01-15T19:08:30.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.212 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.212 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:155 3.fs0 3:f27a3046:::smithi00140323-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:30.212 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:30.212 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.955233376000001, 2021-01-15T19:08:30.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999, 2021-01-15T19:08:30.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.213 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 155 2021-01-15T19:08:30.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.214 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:30.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:30.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9607999999999998e-05 2021-01-15T19:08:30.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237767+0000", 2021-01-15T19:08:30.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3810000000000001e-06 2021-01-15T19:08:30.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237771+0000", 2021-01-15T19:08:30.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5639999999999999e-06 2021-01-15T19:08:30.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237779+0000", 2021-01-15T19:08:30.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.348e-06 2021-01-15T19:08:30.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237782+0000", 2021-01-15T19:08:30.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3377000000000002e-05 2021-01-15T19:08:30.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237816+0000", 2021-01-15T19:08:30.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000003e-06 2021-01-15T19:08:30.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:30.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237822+0000", 2021-01-15T19:08:30.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552276 2021-01-15T19:08:30.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239374+0000", 2021-01-15T19:08:30.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:30.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:30.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239415+0000", 2021-01-15T19:08:30.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010934268 2021-01-15T19:08:30.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250349+0000", 2021-01-15T19:08:30.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011671 2021-01-15T19:08:30.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:30.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250466+0000", 2021-01-15T19:08:30.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050409820000000003 2021-01-15T19:08:30.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255507+0000", 2021-01-15T19:08:30.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0531000000000005e-05 2021-01-15T19:08:30.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255587+0000", 2021-01-15T19:08:30.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122043 2021-01-15T19:08:30.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255709+0000", 2021-01-15T19:08:30.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999 2021-01-15T19:08:30.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.228 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.229 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:156 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~442368] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:30.229 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:30.229 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.951393016999999, 2021-01-15T19:08:30.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759, 2021-01-15T19:08:30.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156 2021-01-15T19:08:30.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.231 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:30.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:30.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030551200000000003 2021-01-15T19:08:30.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241873+0000", 2021-01-15T19:08:30.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6080000000000002e-06 2021-01-15T19:08:30.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241883+0000", 2021-01-15T19:08:30.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241898+0000", 2021-01-15T19:08:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4010000000000002e-06 2021-01-15T19:08:30.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241904+0000", 2021-01-15T19:08:30.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6971999999999997e-05 2021-01-15T19:08:30.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241951+0000", 2021-01-15T19:08:30.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104244 2021-01-15T19:08:30.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242055+0000", 2021-01-15T19:08:30.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070015099999999999 2021-01-15T19:08:30.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242755+0000", 2021-01-15T19:08:30.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0080604460000000006 2021-01-15T19:08:30.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250816+0000", 2021-01-15T19:08:30.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00338733 2021-01-15T19:08:30.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254203+0000", 2021-01-15T19:08:30.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3444999999999998e-05 2021-01-15T19:08:30.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254287+0000", 2021-01-15T19:08:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759 2021-01-15T19:08:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:157 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~401408] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.94858679, 2021-01-15T19:08:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335, 2021-01-15T19:08:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 157 2021-01-15T19:08:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:30.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:30.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000301899 2021-01-15T19:08:30.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244676+0000", 2021-01-15T19:08:30.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6430000000000001e-06 2021-01-15T19:08:30.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244685+0000", 2021-01-15T19:08:30.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5293000000000001e-05 2021-01-15T19:08:30.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244700+0000", 2021-01-15T19:08:30.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6499999999999996e-06 2021-01-15T19:08:30.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244708+0000", 2021-01-15T19:08:30.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1064999999999997e-05 2021-01-15T19:08:30.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244789+0000", 2021-01-15T19:08:30.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117805 2021-01-15T19:08:30.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244906+0000", 2021-01-15T19:08:30.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070339100000000004 2021-01-15T19:08:30.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245610+0000", 2021-01-15T19:08:30.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087607599999999994 2021-01-15T19:08:30.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:30.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107695 2021-01-15T19:08:30.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254478+0000", 2021-01-15T19:08:30.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109134 2021-01-15T19:08:30.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254587+0000", 2021-01-15T19:08:30.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335 2021-01-15T19:08:30.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.257 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.258 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:158 3.cs0 3:3ef45b66:::smithi00140323-11:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:30.258 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:30.258 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.948200773, 2021-01-15T19:08:30.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441, 2021-01-15T19:08:30.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.259 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 158 2021-01-15T19:08:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:30.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5699999999999999e-05 2021-01-15T19:08:30.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244816+0000", 2021-01-15T19:08:30.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6839999999999996e-06 2021-01-15T19:08:30.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244823+0000", 2021-01-15T19:08:30.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4626e-05 2021-01-15T19:08:30.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244838+0000", 2021-01-15T19:08:30.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6500000000000001e-06 2021-01-15T19:08:30.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244844+0000", 2021-01-15T19:08:30.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004205149 2021-01-15T19:08:30.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249049+0000", 2021-01-15T19:08:30.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015840599999999999 2021-01-15T19:08:30.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249207+0000", 2021-01-15T19:08:30.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071027700000000004 2021-01-15T19:08:30.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249918+0000", 2021-01-15T19:08:30.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051411900000000003 2021-01-15T19:08:30.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255059+0000", 2021-01-15T19:08:30.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045701699999999999 2021-01-15T19:08:30.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255516+0000", 2021-01-15T19:08:30.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8710999999999997e-05 2021-01-15T19:08:30.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255614+0000", 2021-01-15T19:08:30.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441 2021-01-15T19:08:30.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.272 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.272 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:159 3.cs0 3:3ef45b66:::smithi00140323-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:30.273 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:30.273 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.948068321999999, 2021-01-15T19:08:30.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195, 2021-01-15T19:08:30.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 159 2021-01-15T19:08:30.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.275 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:30.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:30.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0961000000000003e-05 2021-01-15T19:08:30.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244933+0000", 2021-01-15T19:08:30.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0480000000000001e-06 2021-01-15T19:08:30.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244941+0000", 2021-01-15T19:08:30.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3711e-05 2021-01-15T19:08:30.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244954+0000", 2021-01-15T19:08:30.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2390000000000002e-06 2021-01-15T19:08:30.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244959+0000", 2021-01-15T19:08:30.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082968859999999998 2021-01-15T19:08:30.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253256+0000", 2021-01-15T19:08:30.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000147608 2021-01-15T19:08:30.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:30.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253404+0000", 2021-01-15T19:08:30.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047189329999999998 2021-01-15T19:08:30.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258123+0000", 2021-01-15T19:08:30.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010107000000000001 2021-01-15T19:08:30.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258224+0000", 2021-01-15T19:08:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019277389999999999 2021-01-15T19:08:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260152+0000", 2021-01-15T19:08:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195 2021-01-15T19:08:30.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.287 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:201 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:30.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:30.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.944714995, 2021-01-15T19:08:30.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142, 2021-01-15T19:08:30.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 201 2021-01-15T19:08:30.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:30.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:30.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1060000000000002e-05 2021-01-15T19:08:30.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248307+0000", 2021-01-15T19:08:30.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6740000000000003e-06 2021-01-15T19:08:30.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248316+0000", 2021-01-15T19:08:30.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8746e-05 2021-01-15T19:08:30.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248334+0000", 2021-01-15T19:08:30.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2294e-05 2021-01-15T19:08:30.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248347+0000", 2021-01-15T19:08:30.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6400999999999998e-05 2021-01-15T19:08:30.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248393+0000", 2021-01-15T19:08:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2021-01-15T19:08:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248400+0000", 2021-01-15T19:08:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002761197 2021-01-15T19:08:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251161+0000", 2021-01-15T19:08:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011450099999999999 2021-01-15T19:08:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251275+0000", 2021-01-15T19:08:30.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062315829999999997 2021-01-15T19:08:30.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257507+0000", 2021-01-15T19:08:30.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142 2021-01-15T19:08:30.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:202 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:30.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:30.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.94456162, 2021-01-15T19:08:30.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285, 2021-01-15T19:08:30.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 202 2021-01-15T19:08:30.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.302 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:30.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:30.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5448999999999997e-05 2021-01-15T19:08:30.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248455+0000", 2021-01-15T19:08:30.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:30.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248461+0000", 2021-01-15T19:08:30.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4307e-05 2021-01-15T19:08:30.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248476+0000", 2021-01-15T19:08:30.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5819999999999998e-06 2021-01-15T19:08:30.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248482+0000", 2021-01-15T19:08:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1987e-05 2021-01-15T19:08:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248564+0000", 2021-01-15T19:08:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5850000000000002e-06 2021-01-15T19:08:30.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:30.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248572+0000", 2021-01-15T19:08:30.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045194099999999997 2021-01-15T19:08:30.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253091+0000", 2021-01-15T19:08:30.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020063399999999999 2021-01-15T19:08:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253292+0000", 2021-01-15T19:08:30.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079735020000000004 2021-01-15T19:08:30.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261266+0000", 2021-01-15T19:08:30.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285 2021-01-15T19:08:30.313 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.313 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.313 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.314 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:203 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:30.314 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:30.314 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.944382601999999, 2021-01-15T19:08:30.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438, 2021-01-15T19:08:30.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 203 2021-01-15T19:08:30.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:30.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:30.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8518999999999999e-05 2021-01-15T19:08:30.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248607+0000", 2021-01-15T19:08:30.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8810000000000004e-06 2021-01-15T19:08:30.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248612+0000", 2021-01-15T19:08:30.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5719999999999997e-06 2021-01-15T19:08:30.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248619+0000", 2021-01-15T19:08:30.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5899999999999999e-06 2021-01-15T19:08:30.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248623+0000", 2021-01-15T19:08:30.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5608000000000001e-05 2021-01-15T19:08:30.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248668+0000", 2021-01-15T19:08:30.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0166000000000001e-05 2021-01-15T19:08:30.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:30.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248679+0000", 2021-01-15T19:08:30.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048830590000000004 2021-01-15T19:08:30.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253562+0000", 2021-01-15T19:08:30.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7e-05 2021-01-15T19:08:30.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253659+0000", 2021-01-15T19:08:30.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081539850000000007 2021-01-15T19:08:30.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261813+0000", 2021-01-15T19:08:30.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438 2021-01-15T19:08:30.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.327 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:204 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat,delete] snapc 3=[3] ondisk+write+known_if_redirected e28)", 2021-01-15T19:08:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.944298395000001, 2021-01-15T19:08:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532, 2021-01-15T19:08:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 204 2021-01-15T19:08:30.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.329 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:30.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8349000000000002e-05 2021-01-15T19:08:30.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248701+0000", 2021-01-15T19:08:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0620000000000001e-06 2021-01-15T19:08:30.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248709+0000", 2021-01-15T19:08:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7700000000000001e-06 2021-01-15T19:08:30.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248717+0000", 2021-01-15T19:08:30.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5920000000000001e-06 2021-01-15T19:08:30.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248720+0000", 2021-01-15T19:08:30.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0898e-05 2021-01-15T19:08:30.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248761+0000", 2021-01-15T19:08:30.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-06 2021-01-15T19:08:30.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:30.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248770+0000", 2021-01-15T19:08:30.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052185720000000003 2021-01-15T19:08:30.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253989+0000", 2021-01-15T19:08:30.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105809 2021-01-15T19:08:30.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254095+0000", 2021-01-15T19:08:30.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078837199999999997 2021-01-15T19:08:30.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254883+0000", 2021-01-15T19:08:30.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083722769999999992 2021-01-15T19:08:30.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263255+0000", 2021-01-15T19:08:30.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111839 2021-01-15T19:08:30.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263367+0000", 2021-01-15T19:08:30.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015969899999999999 2021-01-15T19:08:30.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263527+0000", 2021-01-15T19:08:30.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532 2021-01-15T19:08:30.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.344 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:245 3.fs0 3:fae92f17:::smithi00140323-19:head [create,setxattr YOk\\Qsqnjk`Uh[icZasWhcZnad[h[k[Qmk]TihbR (1823) in=40b,setxattr QnfshqY (34) in=7b,setxattr V_tZOrZWhpPekPNs_toQaraRlW\\sWffpapfPROXb (1614) in=40b,setxattr aiTbV]hXcpXjkS]VQonZZkfogbUSnO[bajbZWdRf (839) in=40b,setxattr pZdlecZrV]qp]`sTkf_bSjZVZjhbScXfb[Sr_bsa (18524) in=40b,setxattr kXtOdjofgYOa_Tck\\nYsbVOfgtldphnQ`]P[j]lX (1295) in=40b,setxattr dRpscRhc^OtlbThWSmXVSNge]s (34) in=26b,setxattr T\\s[QdNhlRZh\\\\[X]NpSccmfRcqRaXScZOZ\\ZR^e (1999) in=40b,setxattr YTcfnqlbrRa^rhctrncU_Wa]laamPmYsaqg_f^N` (1471) in=40b,setxattr l_Qeh\\RcafrRYN_YZf_iiaWagZfe[sop]^\\qTggq (561) in=40b,setxattr NShVOQe\\dRYY[tPettNbkgkTVonNshnbYeePWcla (314) in=40b,setxattr ^iVidVQX]Yn`XTY\\pOXm`_PNX`qeckhbQoR`aQWZ (8841) in=40b,setxattr _jkmO]cXrN_S^act\\QOOjpUZOj_Qf^otnZbpWZPU (981) in=40b,setxattr bNbhPX[YP[lppmc`PkWcpnS`XsQNpW\\UYnWr\\_RX (1927) in=40b,setxattr talTebhpYS[[TXcgodoQR\\tf`fVekppjlkWnTqf[ (1753) in=40b,setxattr rp\\XTgd\\X^qaodkhY[elq^RagRhN_NR^j^NmjOUh (806) in=40b,setxattr ]\\oXSo`PWafUO^dbhR^i`hifQgWjRhR[rRa^bnej (70) in=40b,setxattr UkYrdYaZakpnReakOdc\\^bO[YPqfdkdtbNkghaSQ (279) in=40b,setxattr YtUs_hcRdQqgZV]NaadOflcb\\_c^\\Zg_Y\\]snXXZ (701) in=40b,setxattr NRZdh]WclahTe^_btdqe\\^mVYqQOaXaa\\aPVXWnk (19016) in=40b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:30.345 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:30.345 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.8924641539999998, 2021-01-15T19:08:30.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083409910000000007, 2021-01-15T19:08:30.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 245 2021-01-15T19:08:30.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.347 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:30.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:30.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122779 2021-01-15T19:08:30.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300619+0000", 2021-01-15T19:08:30.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0987e-05 2021-01-15T19:08:30.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300630+0000", 2021-01-15T19:08:30.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0562000000000001e-05 2021-01-15T19:08:30.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300651+0000", 2021-01-15T19:08:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329999999999999e-05 2021-01-15T19:08:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300661+0000", 2021-01-15T19:08:30.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6162000000000002e-05 2021-01-15T19:08:30.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300718+0000", 2021-01-15T19:08:30.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105159 2021-01-15T19:08:30.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300823+0000", 2021-01-15T19:08:30.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011057879999999999 2021-01-15T19:08:30.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301928+0000", 2021-01-15T19:08:30.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064543059999999999 2021-01-15T19:08:30.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308383+0000", 2021-01-15T19:08:30.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031878700000000001 2021-01-15T19:08:30.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308702+0000", 2021-01-15T19:08:30.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136131 2021-01-15T19:08:30.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308838+0000", 2021-01-15T19:08:30.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083409910000000007 2021-01-15T19:08:30.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:246 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:30.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:30.360 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.8922495829999999, 2021-01-15T19:08:30.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005, 2021-01-15T19:08:30.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 246 2021-01-15T19:08:30.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.361 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:30.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:30.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1804000000000001e-05 2021-01-15T19:08:30.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300763+0000", 2021-01-15T19:08:30.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5610000000000004e-06 2021-01-15T19:08:30.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300770+0000", 2021-01-15T19:08:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300782+0000", 2021-01-15T19:08:30.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4460000000000003e-06 2021-01-15T19:08:30.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300786+0000", 2021-01-15T19:08:30.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003399596 2021-01-15T19:08:30.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304186+0000", 2021-01-15T19:08:30.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125399 2021-01-15T19:08:30.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304311+0000", 2021-01-15T19:08:30.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057843119999999998 2021-01-15T19:08:30.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310095+0000", 2021-01-15T19:08:30.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005 2021-01-15T19:08:30.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:247 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:30.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:30.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.89213247, 2021-01-15T19:08:30.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281, 2021-01-15T19:08:30.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 247 2021-01-15T19:08:30.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.373 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:30.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:30.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2098999999999997e-05 2021-01-15T19:08:30.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300880+0000", 2021-01-15T19:08:30.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2021-01-15T19:08:30.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300887+0000", 2021-01-15T19:08:30.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.152e-05 2021-01-15T19:08:30.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300898+0000", 2021-01-15T19:08:30.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.904e-06 2021-01-15T19:08:30.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300903+0000", 2021-01-15T19:08:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039909860000000002 2021-01-15T19:08:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304894+0000", 2021-01-15T19:08:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012664060000000001 2021-01-15T19:08:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306161+0000", 2021-01-15T19:08:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044999549999999999 2021-01-15T19:08:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310661+0000", 2021-01-15T19:08:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281 2021-01-15T19:08:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:248 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.8920179419999998, 2021-01-15T19:08:30.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004, 2021-01-15T19:08:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 248 2021-01-15T19:08:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9626999999999998e-05 2021-01-15T19:08:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300993+0000", 2021-01-15T19:08:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1990000000000003e-06 2021-01-15T19:08:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300999+0000", 2021-01-15T19:08:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1737e-05 2021-01-15T19:08:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301010+0000", 2021-01-15T19:08:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.006e-05 2021-01-15T19:08:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301021+0000", 2021-01-15T19:08:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054558510000000003 2021-01-15T19:08:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306476+0000", 2021-01-15T19:08:30.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000193029 2021-01-15T19:08:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306669+0000", 2021-01-15T19:08:30.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046625010000000003 2021-01-15T19:08:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.311332+0000", 2021-01-15T19:08:30.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004 2021-01-15T19:08:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.394 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:252 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~647168] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.884279512, 2021-01-15T19:08:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996, 2021-01-15T19:08:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 252 2021-01-15T19:08:30.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:30.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:30.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041048799999999998 2021-01-15T19:08:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309092+0000", 2021-01-15T19:08:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:08:30.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309101+0000", 2021-01-15T19:08:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4919000000000001e-05 2021-01-15T19:08:30.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309116+0000", 2021-01-15T19:08:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2021-01-15T19:08:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309122+0000", 2021-01-15T19:08:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0148999999999997e-05 2021-01-15T19:08:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309172+0000", 2021-01-15T19:08:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013501799999999999 2021-01-15T19:08:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309307+0000", 2021-01-15T19:08:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015785269999999999 2021-01-15T19:08:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310886+0000", 2021-01-15T19:08:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070233450000000003 2021-01-15T19:08:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.317909+0000", 2021-01-15T19:08:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2075000000000002e-05 2021-01-15T19:08:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318001+0000", 2021-01-15T19:08:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0710999999999994e-05 2021-01-15T19:08:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318072+0000", 2021-01-15T19:08:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996 2021-01-15T19:08:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:285 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~696320] snapc 10=[10] ondisk+write+known_if_redirected e40)", 2021-01-15T19:08:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.0059108669999999, 2021-01-15T19:08:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093549089999999998, 2021-01-15T19:08:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 285 2021-01-15T19:08:30.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041889100000000001 2021-01-15T19:08:30.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187469+0000", 2021-01-15T19:08:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2860999999999999e-05 2021-01-15T19:08:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187482+0000", 2021-01-15T19:08:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1379e-05 2021-01-15T19:08:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187503+0000", 2021-01-15T19:08:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0691e-05 2021-01-15T19:08:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187514+0000", 2021-01-15T19:08:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9081e-05 2021-01-15T19:08:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187563+0000", 2021-01-15T19:08:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8070000000000002e-06 2021-01-15T19:08:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:30.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187570+0000", 2021-01-15T19:08:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019352589999999999 2021-01-15T19:08:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.189505+0000", 2021-01-15T19:08:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011456299999999999 2021-01-15T19:08:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.189620+0000", 2021-01-15T19:08:30.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000972923 2021-01-15T19:08:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190592+0000", 2021-01-15T19:08:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056078150000000004 2021-01-15T19:08:30.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196200+0000", 2021-01-15T19:08:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102021 2021-01-15T19:08:30.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196302+0000", 2021-01-15T19:08:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102618 2021-01-15T19:08:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196405+0000", 2021-01-15T19:08:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093549089999999998 2021-01-15T19:08:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.423 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:287 3.cs0 3:30718dd9:::smithi00140323-34:head [setxattr _header (56) in=7b] snapc 10=[10] ondisk+write+known_if_redirected e40)", 2021-01-15T19:08:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.190355+0000", 2021-01-15T19:08:30.424 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.0026061620000002, 2021-01-15T19:08:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068383029999999996, 2021-01-15T19:08:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.425 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 287 2021-01-15T19:08:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190355+0000", 2021-01-15T19:08:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190355+0000", 2021-01-15T19:08:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.6157e-05 2021-01-15T19:08:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190381+0000", 2021-01-15T19:08:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6219999999999998e-06 2021-01-15T19:08:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190384+0000", 2021-01-15T19:08:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:08:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190391+0000", 2021-01-15T19:08:30.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8219999999999998e-06 2021-01-15T19:08:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190394+0000", 2021-01-15T19:08:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047188990000000004 2021-01-15T19:08:30.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.195113+0000", 2021-01-15T19:08:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012676899999999999 2021-01-15T19:08:30.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.195239+0000", 2021-01-15T19:08:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049535099999999999 2021-01-15T19:08:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.195735+0000", 2021-01-15T19:08:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070372800000000004 2021-01-15T19:08:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196438+0000", 2021-01-15T19:08:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00069528299999999999 2021-01-15T19:08:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.197134+0000", 2021-01-15T19:08:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9280999999999997e-05 2021-01-15T19:08:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.197193+0000", 2021-01-15T19:08:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068383029999999996 2021-01-15T19:08:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:288 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e40)", 2021-01-15T19:08:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.190414+0000", 2021-01-15T19:08:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.0025464450000001, 2021-01-15T19:08:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086577609999999999, 2021-01-15T19:08:30.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 288 2021-01-15T19:08:30.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190414+0000", 2021-01-15T19:08:30.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190414+0000", 2021-01-15T19:08:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8944e-05 2021-01-15T19:08:30.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190433+0000", 2021-01-15T19:08:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3690000000000007e-06 2021-01-15T19:08:30.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190442+0000", 2021-01-15T19:08:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9399999999999996e-06 2021-01-15T19:08:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190449+0000", 2021-01-15T19:08:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2021-01-15T19:08:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190451+0000", 2021-01-15T19:08:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055543759999999998 2021-01-15T19:08:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196006+0000", 2021-01-15T19:08:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111593 2021-01-15T19:08:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196117+0000", 2021-01-15T19:08:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001568949 2021-01-15T19:08:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.197686+0000", 2021-01-15T19:08:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6741000000000002e-05 2021-01-15T19:08:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.197773+0000", 2021-01-15T19:08:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012991770000000001 2021-01-15T19:08:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.199072+0000", 2021-01-15T19:08:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086577609999999999 2021-01-15T19:08:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:296 3.fs0 3:fae92f17:::smithi00140323-19:head [append 0~729088] snapc 10=[10] ondisk+write+known_if_redirected e40)", 2021-01-15T19:08:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.211804+0000", 2021-01-15T19:08:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.9811570279999999, 2021-01-15T19:08:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083155529999999998, 2021-01-15T19:08:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 296 2021-01-15T19:08:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.211804+0000", 2021-01-15T19:08:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.211804+0000", 2021-01-15T19:08:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019096400000000001 2021-01-15T19:08:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.211995+0000", 2021-01-15T19:08:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5130000000000002e-06 2021-01-15T19:08:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212001+0000", 2021-01-15T19:08:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7623000000000001e-05 2021-01-15T19:08:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212019+0000", 2021-01-15T19:08:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1079999999999998e-06 2021-01-15T19:08:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212025+0000", 2021-01-15T19:08:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7986999999999999e-05 2021-01-15T19:08:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212083+0000", 2021-01-15T19:08:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1340000000000003e-05 2021-01-15T19:08:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212174+0000", 2021-01-15T19:08:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00089311700000000002 2021-01-15T19:08:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:30.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.213067+0000", 2021-01-15T19:08:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068268399999999998 2021-01-15T19:08:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.219894+0000", 2021-01-15T19:08:30.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122741 2021-01-15T19:08:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.220017+0000", 2021-01-15T19:08:30.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010232 2021-01-15T19:08:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.220119+0000", 2021-01-15T19:08:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083155529999999998 2021-01-15T19:08:30.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.464 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:299 3.fs0 3:fae92f17:::smithi00140323-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e40)", 2021-01-15T19:08:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.215288+0000", 2021-01-15T19:08:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.977672356, 2021-01-15T19:08:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007619064, 2021-01-15T19:08:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:30.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 299 2021-01-15T19:08:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:30.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.215288+0000", 2021-01-15T19:08:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.215288+0000", 2021-01-15T19:08:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0679000000000001e-05 2021-01-15T19:08:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.215329+0000", 2021-01-15T19:08:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:08:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:30.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.215338+0000", 2021-01-15T19:08:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4392e-05 2021-01-15T19:08:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:30.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.215352+0000", 2021-01-15T19:08:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8570000000000003e-06 2021-01-15T19:08:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:30.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.215357+0000", 2021-01-15T19:08:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048112839999999999 2021-01-15T19:08:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.220168+0000", 2021-01-15T19:08:30.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111526 2021-01-15T19:08:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.220280+0000", 2021-01-15T19:08:30.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015885129999999999 2021-01-15T19:08:30.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:30.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.221868+0000", 2021-01-15T19:08:30.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001059 2021-01-15T19:08:30.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:30.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.221974+0000", 2021-01-15T19:08:30.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093349399999999999 2021-01-15T19:08:30.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:30.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:30.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:30.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.222908+0000", 2021-01-15T19:08:30.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007619064 2021-01-15T19:08:30.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.476 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:30.477 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:30.477 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:30.478 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:30.479 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:30.477+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:30.480 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:30.580 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:30.578+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:30.638 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:30.638 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:30.639 INFO:tasks.thrashosds.thrasher:Growing pool rbd 2021-01-15T19:08:30.640 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:30.655 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:30.655 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:30.656 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:30.656 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:30.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:30.670 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:30.681 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:30.680+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:30.710 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:30.730 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:30.753 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:30.775 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:30.782 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:30.781+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:30.798 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:30.808 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:30.808 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:30.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:30.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:30.841 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:30.843 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.843 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.843 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.844 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.844 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.844 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.844 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.844 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:30.883 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:30.881+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:30.952 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:30.952 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:30.962 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:30.973 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:30.975 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:30.984 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:30.982+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:30.986 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2021-01-15T19:08:30.177170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30646272,"num_objects":61,"num_object_clones":10,"num_object_copies":181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":166,"num_read_kb":15903,"num_write":203,"num_write_kb":31001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":90624,"kb_used_data":48064,"kb_used_omap":0,"kb_used_meta":42496,"kb_avail":754884096,"statfs":{"total":773094113280,"available":773001314304,"internally_reserved":0,"allocated":49217536,"data_stored":54123851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":43515904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2023424,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":-1,"num_read":14,"num_read_kb":4085,"num_write":26,"num_write_kb":1976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.895478"},"pg_stats":[{"pgid":"3.f","version":"38'24","reported_seq":"77","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:26.336135+0000","last_change":"2021-01-15T19:08:26.336135+0000","last_active":"2021-01-15T19:08:26.336135+0000","last_peered":"2021-01-15T19:08:26.336135+0000","last_clean":"2021-01-15T19:08:26.336135+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:26.336135+0000","last_undegraded":"2021-01-15T19:08:26.336135+0000","last_fullsized":"2021-01-15T19:08:26.336135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":2646016,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":2622,"num_write":30,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.e","version":"25'4","reported_seq":"53","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.281162+0000","last_change":"2021-01-15T19:08:25.281562+0000","last_active":"2021-01-15T19:08:25.281162+0000","last_peered":"2021-01-15T19:08:25.281162+0000","last_clean":"2021-01-15T19:08:25.281162+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:25.281162+0000","last_undegraded":"2021-01-15T19:08:25.281162+0000","last_fullsized":"2021-01-15T19:08:25.281162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":868352,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":555,"num_write":4,"num_write_kb":848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.d","version":"35'10","reported_seq":"63","reported_epoch":"38","state":"active+clean","last_fresh":"2021-01-15T19:08:26.283897+0000","last_change":"2021-01-15T19:08:26.283897+0000","last_active":"2021-01-15T19:08:26.283897+0000","last_peered":"2021-01-15T19:08:26.283897+0000","last_clean":"2021-01-15T19:08:26.283897+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:26.283897+0000","last_undegraded":"2021-01-15T19:08:26.283897+0000","last_fullsized":"2021-01-15T19:08:26.283897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":417792,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":220,"num_write":26,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"3"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.c","version":"38'29","reported_seq":"82","reported_epoch":"38","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:26.335637+0000","last_change":"2021-01-15T19:08:26.335637+0000","last_active":"2021-01-15T19:08:26.335637+0000","last_peered":"2021-01-15T19:08:26.335637+0000","last_clean":"2021-01-15T19:08:26.335637+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:26.335637+0000","last_undegraded":"2021-01-15T19:08:26.335637+0000","last_fullsized":"2021-01-15T19:08:26.335637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":4988928,"num_objects":7,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":1641,"num_write":20,"num_write_kb":4872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.b","version":"25'4","reported_seq":"59","reported_epoch":"38","state":"active+clean","last_fresh":"2021-01-15T19:08:26.287106+0000","last_change":"2021-01-15T19:08:26.287106+0000","last_active":"2021-01-15T19:08:26.287106+0000","last_peered":"2021-01-15T19:08:26.287106+0000","last_clean":"2021-01-15T19:08:26.287106+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:26.287106+0000","last_undegraded":"2021-01-15T19:08:26.287106+0000","last_fullsized":"2021-01-15T19:08:26.287106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1220608,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2139,"num_write":4,"num_write_kb":1192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"3"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.a","version":"25'6","reported_seq":"57","reported_epoch":"38","state":"active+clean","last_fresh":"2021-01-15T19:08:26.283886+0000","last_change":"2021-01-15T19:08:26.284214+0000","last_active":"2021-01-15T19:08:26.283886+0000","last_peered":"2021-01-15T19:08:26.283886+0000","last_clean":"2021-01-15T19:08:26.283886+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:26.283886+0000","last_undegraded":"2021-01-15T19:08:26.283886+0000","last_fullsized":"2021-01-15T19:08:26.283886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1843200,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"3"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.9","version":"28'17","reported_seq":"72","reported_epoch":"38","state":"active+clean","last_fresh":"2021-01-15T19:08:26.284377+0000","last_change":"2021-01-15T19:08:26.284737+0000","last_active":"2021-01-15T19:08:26.284377+0000","last_peered":"2021-01-15T19:08:26.284377+0000","last_clean":"2021-01-15T19:08:26.284377+0000","last_became_active":"2021-01-15T19:08:10.230943+0000","last_became_peered":"2021-01-15T19:08:10.230943+0000","last_unstale":"2021-01-15T19:08:26.284377+0000","last_undegraded":"2021-01-15T19:08:26.284377+0000","last_fullsized":"2021-01-15T19:08:26.284377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3129344,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":1831,"num_write":14,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"3"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.8","version":"28'16","reported_seq":"65","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.291517+0000","last_change":"2021-01-15T19:08:25.291861+0000","last_active":"2021-01-15T19:08:25.291517+0000","last_peered":"2021-01-15T19:08:25.291517+0000","last_clean":"2021-01-15T19:08:25.291517+0000","last_became_active":"2021-01-15T19:08:10.231245+0000","last_became_peered":"2021-01-15T19:08:10.231245+0000","last_unstale":"2021-01-15T19:08:25.291517+0000","last_undegraded":"2021-01-15T19:08:25.291517+0000","last_fullsized":"2021-01-15T19:08:25.291517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2547712,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":6,"num_write":12,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"26","reported_epoch":"36","state":"active+clean","last_fresh":"2021-01-15T19:08:24.291517+0000","last_change":"2021-01-15T19:07:56.945822+0000","last_active":"2021-01-15T19:08:24.291517+0000","last_peered":"2021-01-15T19:08:24.291517+0000","last_clean":"2021-01-15T19:08:24.291517+0000","last_became_active":"2021-01-15T19:07:56.945402+0000","last_became_peered":"2021-01-15T19:07:56.945402+0000","last_unstale":"2021-01-15T19:08:24.291517+0000","last_undegraded":"2021-01-15T19:08:24.291517+0000","last_fullsized":"2021-01-15T19:08:24.291517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'6","reported_seq":"52","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.281075+0000","last_change":"2021-01-15T19:08:25.281551+0000","last_active":"2021-01-15T19:08:25.281075+0000","last_peered":"2021-01-15T19:08:25.281075+0000","last_clean":"2021-01-15T19:08:25.281075+0000","last_became_active":"2021-01-15T19:08:10.230610+0000","last_became_peered":"2021-01-15T19:08:10.230610+0000","last_unstale":"2021-01-15T19:08:25.281075+0000","last_undegraded":"2021-01-15T19:08:25.281075+0000","last_fullsized":"2021-01-15T19:08:25.281075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1548288,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"26","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.286144+0000","last_change":"2021-01-15T19:07:56.946100+0000","last_active":"2021-01-15T19:08:25.286144+0000","last_peered":"2021-01-15T19:08:25.286144+0000","last_clean":"2021-01-15T19:08:25.286144+0000","last_became_active":"2021-01-15T19:07:56.945756+0000","last_became_peered":"2021-01-15T19:07:56.945756+0000","last_unstale":"2021-01-15T19:08:25.286144+0000","last_undegraded":"2021-01-15T19:08:25.286144+0000","last_fullsized":"2021-01-15T19:08:25.286144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"47","reported_epoch":"38","state":"active+clean","last_fresh":"2021-01-15T19:08:26.335582+0000","last_change":"2021-01-15T19:08:26.335889+0000","last_active":"2021-01-15T19:08:26.335582+0000","last_peered":"2021-01-15T19:08:26.335582+0000","last_clean":"2021-01-15T19:08:26.335582+0000","last_became_active":"2021-01-15T19:08:10.237812+0000","last_became_peered":"2021-01-15T19:08:10.237812+0000","last_unstale":"2021-01-15T19:08:26.335582+0000","last_undegraded":"2021-01-15T19:08:26.335582+0000","last_fullsized":"2021-01-15T19:08:26.335582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"3"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"37","reported_epoch":"38","state":"active+clean","last_fresh":"2021-01-15T19:08:26.288548+0000","last_change":"2021-01-15T19:08:26.288548+0000","last_active":"2021-01-15T19:08:26.288548+0000","last_peered":"2021-01-15T19:08:26.288548+0000","last_clean":"2021-01-15T19:08:26.288548+0000","last_became_active":"2021-01-15T19:08:26.288148+0000","last_became_peered":"2021-01-15T19:08:26.288148+0000","last_unstale":"2021-01-15T19:08:26.288548+0000","last_undegraded":"2021-01-15T19:08:26.288548+0000","last_fullsized":"2021-01-15T19:08:26.288548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"37'14","reported_seq":"68","reported_epoch":"37","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:25.280628+0000","last_change":"2021-01-15T19:08:25.280628+0000","last_active":"2021-01-15T19:08:25.280628+0000","last_peered":"2021-01-15T19:08:25.280628+0000","last_clean":"2021-01-15T19:08:25.280628+0000","last_became_active":"2021-01-15T19:08:10.231211+0000","last_became_peered":"2021-01-15T19:08:10.231211+0000","last_unstale":"2021-01-15T19:08:25.280628+0000","last_undegraded":"2021-01-15T19:08:25.280628+0000","last_fullsized":"2021-01-15T19:08:25.280628+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":1523712,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":22,"num_read_kb":1928,"num_write":11,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":"26","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.285871+0000","last_change":"2021-01-15T19:07:56.182516+0000","last_active":"2021-01-15T19:08:25.285871+0000","last_peered":"2021-01-15T19:08:25.285871+0000","last_clean":"2021-01-15T19:08:25.285871+0000","last_became_active":"2021-01-15T19:07:56.182068+0000","last_became_peered":"2021-01-15T19:07:56.182068+0000","last_unstale":"2021-01-15T19:08:25.285871+0000","last_undegraded":"2021-01-15T19:08:25.285871+0000","last_fullsized":"2021-01-15T19:08:25.285871+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"33","reported_epoch":"36","state":"active+clean","last_fresh":"2021-01-15T19:08:24.305223+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:24.305223+0000","last_peered":"2021-01-15T19:08:24.305223+0000","last_clean":"2021-01-15T19:08:24.305223+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:24.305223+0000","last_undegraded":"2021-01-15T19:08:24.305223+0000","last_fullsized":"2021-01-15T19:08:24.305223+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"28'12","reported_seq":"62","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.291580+0000","last_change":"2021-01-15T19:08:25.291815+0000","last_active":"2021-01-15T19:08:25.291580+0000","last_peered":"2021-01-15T19:08:25.291580+0000","last_clean":"2021-01-15T19:08:25.291580+0000","last_became_active":"2021-01-15T19:08:10.234192+0000","last_became_peered":"2021-01-15T19:08:10.234192+0000","last_unstale":"2021-01-15T19:08:25.291580+0000","last_undegraded":"2021-01-15T19:08:25.291580+0000","last_fullsized":"2021-01-15T19:08:25.291580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1974272,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3221,"num_write":9,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"26","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.285866+0000","last_change":"2021-01-15T19:07:56.185113+0000","last_active":"2021-01-15T19:08:25.285866+0000","last_peered":"2021-01-15T19:08:25.285866+0000","last_clean":"2021-01-15T19:08:25.285866+0000","last_became_active":"2021-01-15T19:07:56.184624+0000","last_became_peered":"2021-01-15T19:07:56.184624+0000","last_unstale":"2021-01-15T19:08:25.285866+0000","last_undegraded":"2021-01-15T19:08:25.285866+0000","last_fullsized":"2021-01-15T19:08:25.285866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"28'16","reported_seq":"63","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.303834+0000","last_change":"2021-01-15T19:08:25.304058+0000","last_active":"2021-01-15T19:08:25.303834+0000","last_peered":"2021-01-15T19:08:25.303834+0000","last_clean":"2021-01-15T19:08:25.303834+0000","last_became_active":"2021-01-15T19:08:10.230297+0000","last_became_peered":"2021-01-15T19:08:10.230297+0000","last_unstale":"2021-01-15T19:08:25.303834+0000","last_undegraded":"2021-01-15T19:08:25.303834+0000","last_fullsized":"2021-01-15T19:08:25.303834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1966080,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":1314,"num_write":29,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"26","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.286372+0000","last_change":"2021-01-15T19:07:56.185160+0000","last_active":"2021-01-15T19:08:25.286372+0000","last_peered":"2021-01-15T19:08:25.286372+0000","last_clean":"2021-01-15T19:08:25.286372+0000","last_became_active":"2021-01-15T19:07:56.184648+0000","last_became_peered":"2021-01-15T19:07:56.184648+0000","last_unstale":"2021-01-15T19:08:25.286372+0000","last_undegraded":"2021-01-15T19:08:25.286372+0000","last_fullsized":"2021-01-15T19:08:25.286372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"37'18","reported_seq":"62","reported_epoch":"37","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:25.303887+0000","last_change":"2021-01-15T19:08:25.303887+0000","last_active":"2021-01-15T19:08:25.303887+0000","last_peered":"2021-01-15T19:08:25.303887+0000","last_clean":"2021-01-15T19:08:25.303887+0000","last_became_active":"2021-01-15T19:08:10.228436+0000","last_became_peered":"2021-01-15T19:08:10.228436+0000","last_unstale":"2021-01-15T19:08:25.303887+0000","last_undegraded":"2021-01-15T19:08:25.303887+0000","last_fullsized":"2021-01-15T19:08:25.303887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":3178496,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":12,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"}]},{"pgid":"3.7","version":"35'16","reported_seq":"60","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.291896+0000","last_change":"2021-01-15T19:08:25.292212+0000","last_active":"2021-01-15T19:08:25.291896+0000","last_peered":"2021-01-15T19:08:25.291896+0000","last_clean":"2021-01-15T19:08:25.291896+0000","last_became_active":"2021-01-15T19:08:10.703898+0000","last_became_peered":"2021-01-15T19:08:10.703898+0000","last_unstale":"2021-01-15T19:08:25.291896+0000","last_undegraded":"2021-01-15T19:08:25.291896+0000","last_fullsized":"2021-01-15T19:08:25.291896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1318912,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2021-01-15T19:08:24.288973+0000","last_change":"2021-01-15T19:07:56.779451+0000","last_active":"2021-01-15T19:08:24.288973+0000","last_peered":"2021-01-15T19:08:24.288973+0000","last_clean":"2021-01-15T19:08:24.288973+0000","last_became_active":"2021-01-15T19:07:56.779138+0000","last_became_peered":"2021-01-15T19:07:56.779138+0000","last_unstale":"2021-01-15T19:08:24.288973+0000","last_undegraded":"2021-01-15T19:08:24.288973+0000","last_fullsized":"2021-01-15T19:08:24.288973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"26","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.286107+0000","last_change":"2021-01-15T19:07:56.779540+0000","last_active":"2021-01-15T19:08:25.286107+0000","last_peered":"2021-01-15T19:08:25.286107+0000","last_clean":"2021-01-15T19:08:25.286107+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:08:25.286107+0000","last_undegraded":"2021-01-15T19:08:25.286107+0000","last_fullsized":"2021-01-15T19:08:25.286107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"28'13","reported_seq":"63","reported_epoch":"37","state":"active+clean","last_fresh":"2021-01-15T19:08:25.291983+0000","last_change":"2021-01-15T19:08:25.292219+0000","last_active":"2021-01-15T19:08:25.291983+0000","last_peered":"2021-01-15T19:08:25.291983+0000","last_clean":"2021-01-15T19:08:25.291983+0000","last_became_active":"2021-01-15T19:08:10.703953+0000","last_became_peered":"2021-01-15T19:08:10.703953+0000","last_unstale":"2021-01-15T19:08:25.291983+0000","last_undegraded":"2021-01-15T19:08:25.291983+0000","last_fullsized":"2021-01-15T19:08:25.291983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1474560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":413,"num_write":10,"num_write_kb":1592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"5","length":"2"},{"start":"9","length":"1"},{"start":"c","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":30646272,"num_objects":59,"num_object_clones":10,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":1,"num_read":166,"num_read_kb":15903,"num_write":201,"num_write_kb":30992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":46391296,"data_stored":52482048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12300,"kb_used_data":7044,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":94359540,"statfs":{"total":96636764160,"available":96624168960,"internally_reserved":0,"allocated":7213056,"data_stored":7617524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.415}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10220,"kb_used_data":4900,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94361620,"statfs":{"total":96636764160,"available":96626298880,"internally_reserved":0,"allocated":5017600,"data_stored":5456355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9472,"kb_used_data":4152,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94362368,"statfs":{"total":96636764160,"available":96627064832,"internally_reserved":0,"allocated":4251648,"data_stored":4968931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11436,"kb_used_data":6116,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94360404,"statfs":{"total":96636764160,"available":96625053696,"internally_reserved":0,"allocated":6262784,"data_stored":7093236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12396,"kb_used_data":7012,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94359444,"statfs":{"total":96636764160,"available":96624070656,"internally_reserved":0,"allocated":7180288,"data_stored":7422435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10664,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":94361176,"statfs":{"total":96636764160,"available":96625844224,"internally_reserved":0,"allocated":5603328,"data_stored":5811188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15960,"kb_used_data":10256,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94355880,"statfs":{"total":96636764160,"available":96620421120,"internally_reserved":0,"allocated":10502144,"data_stored":12239331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":3186688,"data_stored":3514851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]}]}]}} 2021-01-15T19:08:30.989 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2021-01-15T19:08:30.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set rbd pg_num 18 2021-01-15T19:08:31.084 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:31.082+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:31.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:31.181 INFO:tasks.rados.rados.0.smithi001.stdout:131: done (1 left) 2021-01-15T19:08:31.181 INFO:tasks.rados.rados.0.smithi001.stdout:132: rmattr oid 31 current snap is 10 2021-01-15T19:08:31.182 INFO:tasks.rados.rados.0.smithi001.stdout:132: done (1 left) 2021-01-15T19:08:31.182 INFO:tasks.rados.rados.0.smithi001.stdout:133: read oid 13 snap -1 2021-01-15T19:08:31.182 INFO:tasks.rados.rados.0.smithi001.stdout:133: expect (ObjNum 12 snap 0 seq_num 12) 2021-01-15T19:08:31.182 INFO:tasks.rados.rados.0.smithi001.stdout:134: rollback oid 14 current snap is 10 2021-01-15T19:08:31.183 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 14 to 7 2021-01-15T19:08:31.183 INFO:tasks.rados.rados.0.smithi001.stdout:130: expect (ObjNum 43 snap 0 seq_num 43) 2021-01-15T19:08:31.185 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:31.184+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:08:31.204 INFO:tasks.rados.rados.0.smithi001.stdout:134: finishing rollback tid 1 to smithi00140323-14 2021-01-15T19:08:31.204 INFO:tasks.rados.rados.0.smithi001.stdout:134: finishing rollback tid 2 to smithi00140323-14 2021-01-15T19:08:31.205 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 35 (ObjNum 55 snap 2 seq_num 55) dirty exists 2021-01-15T19:08:31.205 INFO:tasks.rados.rados.0.smithi001.stdout:133: expect (ObjNum 12 snap 0 seq_num 12) 2021-01-15T19:08:31.209 INFO:tasks.rados.rados.0.smithi001.stdout:130: done (2 left) 2021-01-15T19:08:31.209 INFO:tasks.rados.rados.0.smithi001.stdout:133: done (1 left) 2021-01-15T19:08:31.209 INFO:tasks.rados.rados.0.smithi001.stdout:134: done (0 left) 2021-01-15T19:08:31.210 INFO:tasks.rados.rados.0.smithi001.stdout:135: snap_create 2021-01-15T19:08:31.249 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:31.249 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:31.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:31.286 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:31.284+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:31.387 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:31.385+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:31.433 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:31.434 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:31.443 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:31.488 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:31.486+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:31.589 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:31.587+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:31.615 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:31.615 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:31.625 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:31.690 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:31.688+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:31.758 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:31.758 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:31.768 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:31.812 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:31.811+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:31.894 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:31.892+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:31.900 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:31.900 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:31.910 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:31.999 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:31.998+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:32.049 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.050 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.060 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:32.098 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:32.096+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:32.187 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.187 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.189 INFO:tasks.rados.rados.0.smithi001.stdout:135: done (0 left) 2021-01-15T19:08:32.189 INFO:tasks.rados.rados.0.smithi001.stdout:136: read oid 10 snap -1 2021-01-15T19:08:32.190 INFO:tasks.rados.rados.0.smithi001.stdout:136: expect deleted 2021-01-15T19:08:32.190 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 29 current snap is 11 2021-01-15T19:08:32.190 INFO:tasks.rados.rados.0.smithi001.stdout:137: seq_num 74 ranges {475136=442368,917504=696320,1613824=106496} 2021-01-15T19:08:32.192 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 2 pg_num to 18 2021-01-15T19:08:32.194 INFO:tasks.rados.rados.0.smithi001.stdout:137: writing smithi00140323-29 from 475136 to 917504 tid 1 2021-01-15T19:08:32.200 INFO:tasks.rados.rados.0.smithi001.stdout:137: writing smithi00140323-29 from 917504 to 1613824 tid 2 2021-01-15T19:08:32.204 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:32.205 INFO:tasks.rados.rados.0.smithi001.stdout:137: writing smithi00140323-29 from 1613824 to 1720320 tid 3 2021-01-15T19:08:32.205 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 47 current snap is 11 2021-01-15T19:08:32.206 INFO:tasks.rados.rados.0.smithi001.stdout:138: seq_num 75 ranges {1228800=458752,1687552=581632,2269184=221184} 2021-01-15T19:08:32.206 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:32.199+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:32.208 INFO:tasks.rados.rados.0.smithi001.stdout:138: writing smithi00140323-47 from 1228800 to 1687552 tid 1 2021-01-15T19:08:32.213 INFO:tasks.rados.rados.0.smithi001.stdout:138: writing smithi00140323-47 from 1687552 to 2269184 tid 2 2021-01-15T19:08:32.215 INFO:tasks.rados.rados.0.smithi001.stdout:138: writing smithi00140323-47 from 2269184 to 2490368 tid 3 2021-01-15T19:08:32.215 INFO:tasks.rados.rados.0.smithi001.stdout:139: snap_create 2021-01-15T19:08:32.215 INFO:tasks.rados.rados.0.smithi001.stdout:137: finishing write tid 1 to smithi00140323-29 2021-01-15T19:08:32.216 INFO:tasks.rados.rados.0.smithi001.stdout:137: finishing write tid 2 to smithi00140323-29 2021-01-15T19:08:32.216 INFO:tasks.rados.rados.0.smithi001.stdout:137: finishing write tid 3 to smithi00140323-29 2021-01-15T19:08:32.216 INFO:tasks.rados.rados.0.smithi001.stdout:137: finishing write tid 4 to smithi00140323-29 2021-01-15T19:08:32.216 INFO:tasks.rados.rados.0.smithi001.stdout:137: finishing write tid 5 to smithi00140323-29 2021-01-15T19:08:32.216 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 21 (ObjNum 74 snap 11 seq_num 74) dirty exists 2021-01-15T19:08:32.217 INFO:tasks.rados.rados.0.smithi001.stdout:137: left oid 29 (ObjNum 74 snap 11 seq_num 74) 2021-01-15T19:08:32.217 INFO:tasks.rados.rados.0.smithi001.stdout:138: finishing write tid 1 to smithi00140323-47 2021-01-15T19:08:32.219 INFO:tasks.rados.rados.0.smithi001.stdout:138: finishing write tid 2 to smithi00140323-47 2021-01-15T19:08:32.220 INFO:tasks.rados.rados.0.smithi001.stdout:138: finishing write tid 3 to smithi00140323-47 2021-01-15T19:08:32.220 INFO:tasks.rados.rados.0.smithi001.stdout:138: finishing write tid 4 to smithi00140323-47 2021-01-15T19:08:32.224 INFO:tasks.rados.rados.0.smithi001.stdout:138: finishing write tid 5 to smithi00140323-47 2021-01-15T19:08:32.225 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 31 (ObjNum 75 snap 11 seq_num 75) dirty exists 2021-01-15T19:08:32.226 INFO:tasks.rados.rados.0.smithi001.stdout:138: left oid 47 (ObjNum 75 snap 11 seq_num 75) 2021-01-15T19:08:32.299 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:32.298+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:32.334 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.335 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.344 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:32.402 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:32.400+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:32.476 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.477 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.486 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:32.502 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:32.501+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:32.602 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:32.600+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:32.615 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.615 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.625 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:32.702 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:32.700+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:32.751 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.752 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:32.803 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:32.801+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:32.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:32.904 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:32.901+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:32.927 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:32.927 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:32.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:33.017 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:33.026 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:33.023+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:33.036 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:33.056 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:33.075 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:33.097 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:33.108 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:33.105+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:33.109 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:33.109 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:33.117 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:33.118 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:33.138 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:33.159 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:33.160 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.161 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.161 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.161 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.161 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:33.174 INFO:tasks.rados.rados.0.smithi001.stdout:136: done (3 left) 2021-01-15T19:08:33.175 INFO:tasks.rados.rados.0.smithi001.stdout:137: done (2 left) 2021-01-15T19:08:33.175 INFO:tasks.rados.rados.0.smithi001.stdout:138: done (1 left) 2021-01-15T19:08:33.175 INFO:tasks.rados.rados.0.smithi001.stdout:139: done (0 left) 2021-01-15T19:08:33.175 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 12 2021-01-15T19:08:33.176 INFO:tasks.rados.rados.0.smithi001.stdout:140: seq_num 76 ranges {1761280=475136,2236416=458752,2695168=344064} 2021-01-15T19:08:33.186 INFO:tasks.rados.rados.0.smithi001.stdout:140: writing smithi00140323-39 from 1761280 to 2236416 tid 1 2021-01-15T19:08:33.189 INFO:tasks.rados.rados.0.smithi001.stdout:140: writing smithi00140323-39 from 2236416 to 2695168 tid 2 2021-01-15T19:08:33.192 INFO:tasks.rados.rados.0.smithi001.stdout:140: writing smithi00140323-39 from 2695168 to 3039232 tid 3 2021-01-15T19:08:33.192 INFO:tasks.rados.rados.0.smithi001.stdout:141: read oid 11 snap 10 2021-01-15T19:08:33.192 INFO:tasks.rados.rados.0.smithi001.stdout:141: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:08:33.193 INFO:tasks.rados.rados.0.smithi001.stdout:142: copy_from oid 22 from oid 41 current snap is 12 2021-01-15T19:08:33.193 INFO:tasks.rados.rados.0.smithi001.stdout:143: copy_from oid 47 from oid 37 current snap is 12 2021-01-15T19:08:33.193 INFO:tasks.rados.rados.0.smithi001.stdout:144: rollback oid 21 current snap is 12 2021-01-15T19:08:33.194 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 21 to 9 2021-01-15T19:08:33.194 INFO:tasks.rados.rados.0.smithi001.stdout:145: snap_create 2021-01-15T19:08:33.197 INFO:tasks.rados.rados.0.smithi001.stdout:140: finishing write tid 1 to smithi00140323-39 2021-01-15T19:08:33.199 INFO:tasks.rados.rados.0.smithi001.stdout:140: finishing write tid 2 to smithi00140323-39 2021-01-15T19:08:33.200 INFO:tasks.rados.rados.0.smithi001.stdout:141: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:08:33.203 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:08:33.208 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:33.205+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:33.212 INFO:tasks.rados.rados.0.smithi001.stdout:144: finishing rollback tid 0 to smithi00140323-21 2021-01-15T19:08:33.213 INFO:tasks.rados.rados.0.smithi001.stdout:144: finishing rollback tid 1 to smithi00140323-21 2021-01-15T19:08:33.213 INFO:tasks.rados.rados.0.smithi001.stdout:144: finishing rollback tid 2 to smithi00140323-21 2021-01-15T19:08:33.213 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 12 (ObjNum 20 snap 0 seq_num 20) dirty exists 2021-01-15T19:08:33.213 INFO:tasks.rados.rados.0.smithi001.stdout:140: finishing write tid 3 to smithi00140323-39 2021-01-15T19:08:33.214 INFO:tasks.rados.rados.0.smithi001.stdout:140: finishing write tid 4 to smithi00140323-39 2021-01-15T19:08:33.214 INFO:tasks.rados.rados.0.smithi001.stdout:140: finishing write tid 5 to smithi00140323-39 2021-01-15T19:08:33.214 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 26 (ObjNum 76 snap 12 seq_num 76) dirty exists 2021-01-15T19:08:33.214 INFO:tasks.rados.rados.0.smithi001.stdout:140: left oid 39 (ObjNum 76 snap 12 seq_num 76) 2021-01-15T19:08:33.215 INFO:tasks.rados.rados.0.smithi001.stdout:142: finishing copy_from to smithi00140323-22 2021-01-15T19:08:33.215 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 22 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2021-01-15T19:08:33.215 INFO:tasks.rados.rados.0.smithi001.stdout:142: finishing copy_from racing read to smithi00140323-22 2021-01-15T19:08:33.218 INFO:tasks.rados.rados.0.smithi001.stdout:143: finishing copy_from to smithi00140323-47 2021-01-15T19:08:33.218 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2021-01-15T19:08:33.221 INFO:tasks.rados.rados.0.smithi001.stdout:143: finishing copy_from racing read to smithi00140323-47 2021-01-15T19:08:33.293 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:33.294 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:33.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:33.294 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:33.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.295 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=71, reqid=client.4418.0:183, at_version=27'8, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:33.295 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:33.295 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.039324247, 2021-01-15T19:08:33.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003, 2021-01-15T19:08:33.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:33.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:33.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190392 2021-01-15T19:08:33.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251887+0000", 2021-01-15T19:08:33.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6526999999999999e-05 2021-01-15T19:08:33.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251914+0000", 2021-01-15T19:08:33.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8068999999999999e-05 2021-01-15T19:08:33.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251932+0000", 2021-01-15T19:08:33.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7269999999999996e-06 2021-01-15T19:08:33.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251940+0000", 2021-01-15T19:08:33.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012910800000000001 2021-01-15T19:08:33.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252070+0000", 2021-01-15T19:08:33.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2417e-05 2021-01-15T19:08:33.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252082+0000", 2021-01-15T19:08:33.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050434080000000001 2021-01-15T19:08:33.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257125+0000", 2021-01-15T19:08:33.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6032999999999998e-05 2021-01-15T19:08:33.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257171+0000", 2021-01-15T19:08:33.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003 2021-01-15T19:08:33.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:33.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:33.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.035472888000001, 2021-01-15T19:08:33.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:33.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.311 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:33.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.311 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:33.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:33.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:33.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:33.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:33.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:33.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:33.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:33.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:33.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:33.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:33.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:33.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:33.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=79, reqid=client.4418.0:185, at_version=27'10, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.034277929000002, 2021-01-15T19:08:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999, 2021-01-15T19:08:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5377000000000001e-05 2021-01-15T19:08:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256768+0000", 2021-01-15T19:08:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3387000000000001e-05 2021-01-15T19:08:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256782+0000", 2021-01-15T19:08:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6764999999999998e-05 2021-01-15T19:08:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256819+0000", 2021-01-15T19:08:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256826+0000", 2021-01-15T19:08:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9475999999999998e-05 2021-01-15T19:08:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256886+0000", 2021-01-15T19:08:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1929e-05 2021-01-15T19:08:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256898+0000", 2021-01-15T19:08:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055282589999999998 2021-01-15T19:08:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262426+0000", 2021-01-15T19:08:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2537999999999997e-05 2021-01-15T19:08:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262479+0000", 2021-01-15T19:08:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999 2021-01-15T19:08:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.032746247999999, 2021-01-15T19:08:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.031822039000001, 2021-01-15T19:08:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.031598076000002, 2021-01-15T19:08:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=81, reqid=client.4418.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=27'10))", 2021-01-15T19:08:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.028021969000001, 2021-01-15T19:08:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002, 2021-01-15T19:08:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4539999999999999e-05 2021-01-15T19:08:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263024+0000", 2021-01-15T19:08:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9600000000000006e-06 2021-01-15T19:08:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263032+0000", 2021-01-15T19:08:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2021-01-15T19:08:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263041+0000", 2021-01-15T19:08:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3239999999999996e-06 2021-01-15T19:08:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263046+0000", 2021-01-15T19:08:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4546999999999999e-05 2021-01-15T19:08:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263090+0000", 2021-01-15T19:08:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0596e-05 2021-01-15T19:08:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263101+0000", 2021-01-15T19:08:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052094810000000002 2021-01-15T19:08:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268310+0000", 2021-01-15T19:08:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4725000000000003e-05 2021-01-15T19:08:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268355+0000", 2021-01-15T19:08:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002 2021-01-15T19:08:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.027735491000001, 2021-01-15T19:08:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999, 2021-01-15T19:08:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3807000000000001e-05 2021-01-15T19:08:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263309+0000", 2021-01-15T19:08:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263317+0000", 2021-01-15T19:08:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2021-01-15T19:08:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263327+0000", 2021-01-15T19:08:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9009999999999997e-06 2021-01-15T19:08:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263334+0000", 2021-01-15T19:08:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052278899999999998 2021-01-15T19:08:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268562+0000", 2021-01-15T19:08:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4100000000000001e-05 2021-01-15T19:08:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268576+0000", 2021-01-15T19:08:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999 2021-01-15T19:08:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0072390430000002, 2021-01-15T19:08:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0050598389999994, 2021-01-15T19:08:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0043054750000007, 2021-01-15T19:08:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0041853090000004, 2021-01-15T19:08:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:318 3.1s0 3:8b30e322:::smithi00140323-24:head [copy-from ver 8] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0949847119999996, 2021-01-15T19:08:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729, 2021-01-15T19:08:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 318 2021-01-15T19:08:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2021-01-15T19:08:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196102+0000", 2021-01-15T19:08:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6286e-05 2021-01-15T19:08:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196119+0000", 2021-01-15T19:08:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0645e-05 2021-01-15T19:08:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196139+0000", 2021-01-15T19:08:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576000000000001e-05 2021-01-15T19:08:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196151+0000", 2021-01-15T19:08:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017478200000000001 2021-01-15T19:08:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196326+0000", 2021-01-15T19:08:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019917200000000001 2021-01-15T19:08:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196525+0000", 2021-01-15T19:08:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099001019999999992 2021-01-15T19:08:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.206425+0000", 2021-01-15T19:08:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043897240000000002 2021-01-15T19:08:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210815+0000", 2021-01-15T19:08:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023466619999999998 2021-01-15T19:08:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213161+0000", 2021-01-15T19:08:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108811 2021-01-15T19:08:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213270+0000", 2021-01-15T19:08:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729 2021-01-15T19:08:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:319 3.1s0 3:8b30e322:::smithi00140323-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0948217939999996, 2021-01-15T19:08:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001, 2021-01-15T19:08:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 319 2021-01-15T19:08:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6667e-05 2021-01-15T19:08:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196236+0000", 2021-01-15T19:08:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196243+0000", 2021-01-15T19:08:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4992e-05 2021-01-15T19:08:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196258+0000", 2021-01-15T19:08:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6110000000000001e-06 2021-01-15T19:08:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196264+0000", 2021-01-15T19:08:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046740399999999999 2021-01-15T19:08:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196731+0000", 2021-01-15T19:08:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1105000000000004e-05 2021-01-15T19:08:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196812+0000", 2021-01-15T19:08:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011315451000000001 2021-01-15T19:08:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208128+0000", 2021-01-15T19:08:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010873 2021-01-15T19:08:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208237+0000", 2021-01-15T19:08:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058576419999999997 2021-01-15T19:08:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214094+0000", 2021-01-15T19:08:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119562 2021-01-15T19:08:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214214+0000", 2021-01-15T19:08:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012588000000000001 2021-01-15T19:08:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214340+0000", 2021-01-15T19:08:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001 2021-01-15T19:08:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:323 3.1s0 3:80f33866:::smithi00140323-14:head [delete] snapc 11=[11,10] ondisk+write+peerstat_old+known_if_redirected e41)", 2021-01-15T19:08:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0947131680000002, 2021-01-15T19:08:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061698250000000003, 2021-01-15T19:08:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 323 2021-01-15T19:08:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6786999999999999e-05 2021-01-15T19:08:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196345+0000", 2021-01-15T19:08:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6189999999999996e-06 2021-01-15T19:08:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196351+0000", 2021-01-15T19:08:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1976e-05 2021-01-15T19:08:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196363+0000", 2021-01-15T19:08:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9249999999999995e-06 2021-01-15T19:08:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196371+0000", 2021-01-15T19:08:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047688599999999999 2021-01-15T19:08:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196848+0000", 2021-01-15T19:08:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010886 2021-01-15T19:08:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196957+0000", 2021-01-15T19:08:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087102499999999999 2021-01-15T19:08:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197828+0000", 2021-01-15T19:08:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022233589999999998 2021-01-15T19:08:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.200051+0000", 2021-01-15T19:08:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023389800000000001 2021-01-15T19:08:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202390+0000", 2021-01-15T19:08:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7408000000000001e-05 2021-01-15T19:08:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202478+0000", 2021-01-15T19:08:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061698250000000003 2021-01-15T19:08:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=127, reqid=client.4418.0:362, at_version=45'23, trim_to=0'0, roll_forward_to=39'21))", 2021-01-15T19:08:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.188586+0000", 2021-01-15T19:08:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.102434469, 2021-01-15T19:08:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050624770000000001, 2021-01-15T19:08:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188586+0000", 2021-01-15T19:08:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188586+0000", 2021-01-15T19:08:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018245800000000001 2021-01-15T19:08:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188769+0000", 2021-01-15T19:08:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0593000000000003e-05 2021-01-15T19:08:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188800+0000", 2021-01-15T19:08:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3529000000000001e-05 2021-01-15T19:08:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188823+0000", 2021-01-15T19:08:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2021-01-15T19:08:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188831+0000", 2021-01-15T19:08:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8584e-05 2021-01-15T19:08:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188890+0000", 2021-01-15T19:08:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6521e-05 2021-01-15T19:08:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188906+0000", 2021-01-15T19:08:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046927840000000002 2021-01-15T19:08:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.193599+0000", 2021-01-15T19:08:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9928e-05 2021-01-15T19:08:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.193649+0000", 2021-01-15T19:08:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050624770000000001 2021-01-15T19:08:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=129, reqid=client.4418.0:364, at_version=45'25, trim_to=0'0, roll_forward_to=39'21))", 2021-01-15T19:08:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.094220951999999997, 2021-01-15T19:08:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008030611, 2021-01-15T19:08:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015823399999999999 2021-01-15T19:08:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196958+0000", 2021-01-15T19:08:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1186999999999999e-05 2021-01-15T19:08:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196979+0000", 2021-01-15T19:08:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5041999999999999e-05 2021-01-15T19:08:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196994+0000", 2021-01-15T19:08:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7460000000000003e-06 2021-01-15T19:08:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197001+0000", 2021-01-15T19:08:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021975599999999999 2021-01-15T19:08:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197221+0000", 2021-01-15T19:08:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2945999999999999e-05 2021-01-15T19:08:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197234+0000", 2021-01-15T19:08:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075394920000000001 2021-01-15T19:08:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204773+0000", 2021-01-15T19:08:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7207999999999998e-05 2021-01-15T19:08:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204831+0000", 2021-01-15T19:08:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008030611 2021-01-15T19:08:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s1 45/21 ECSubWrite(tid=35, reqid=client.4418.0:374, at_version=45'10, trim_to=0'0, roll_forward_to=40'8))", 2021-01-15T19:08:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198660+0000", 2021-01-15T19:08:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.092360492000000002, 2021-01-15T19:08:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005233054, 2021-01-15T19:08:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198660+0000", 2021-01-15T19:08:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198660+0000", 2021-01-15T19:08:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5329000000000001e-05 2021-01-15T19:08:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198686+0000", 2021-01-15T19:08:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4364e-05 2021-01-15T19:08:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198700+0000", 2021-01-15T19:08:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:08:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198709+0000", 2021-01-15T19:08:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5260000000000004e-06 2021-01-15T19:08:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198713+0000", 2021-01-15T19:08:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2893999999999999e-05 2021-01-15T19:08:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198766+0000", 2021-01-15T19:08:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2265e-05 2021-01-15T19:08:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198778+0000", 2021-01-15T19:08:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050576299999999996 2021-01-15T19:08:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.203836+0000", 2021-01-15T19:08:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7627000000000002e-05 2021-01-15T19:08:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.203894+0000", 2021-01-15T19:08:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005233054 2021-01-15T19:08:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s1 45/21 ECSubWrite(tid=37, reqid=client.4418.0:375, at_version=45'11, trim_to=0'0, roll_forward_to=40'8))", 2021-01-15T19:08:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198747+0000", 2021-01-15T19:08:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.092273761999999995, 2021-01-15T19:08:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054298929999999999, 2021-01-15T19:08:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198747+0000", 2021-01-15T19:08:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198747+0000", 2021-01-15T19:08:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7022e-05 2021-01-15T19:08:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198774+0000", 2021-01-15T19:08:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0645999999999999e-05 2021-01-15T19:08:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198785+0000", 2021-01-15T19:08:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5989999999999996e-06 2021-01-15T19:08:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198792+0000", 2021-01-15T19:08:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.965e-06 2021-01-15T19:08:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198795+0000", 2021-01-15T19:08:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077181099999999996 2021-01-15T19:08:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.199567+0000", 2021-01-15T19:08:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3137e-05 2021-01-15T19:08:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.199580+0000", 2021-01-15T19:08:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045474110000000003 2021-01-15T19:08:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204128+0000", 2021-01-15T19:08:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9302000000000003e-05 2021-01-15T19:08:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204177+0000", 2021-01-15T19:08:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054298929999999999 2021-01-15T19:08:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=130, reqid=client.4418.0:365, at_version=45'26, trim_to=0'0, roll_forward_to=45'23))", 2021-01-15T19:08:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.092172385999999995, 2021-01-15T19:08:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060228970000000001, 2021-01-15T19:08:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7685e-05 2021-01-15T19:08:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198876+0000", 2021-01-15T19:08:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2933e-05 2021-01-15T19:08:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198889+0000", 2021-01-15T19:08:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5310000000000007e-06 2021-01-15T19:08:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198899+0000", 2021-01-15T19:08:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:08:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198904+0000", 2021-01-15T19:08:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7002999999999997e-05 2021-01-15T19:08:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198951+0000", 2021-01-15T19:08:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2021-01-15T19:08:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198963+0000", 2021-01-15T19:08:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058719660000000002 2021-01-15T19:08:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204835+0000", 2021-01-15T19:08:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6568e-05 2021-01-15T19:08:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204871+0000", 2021-01-15T19:08:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060228970000000001 2021-01-15T19:08:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:33.583 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:33.584 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:33.582+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:33.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:33.685 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:33.684+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:33.751 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:33.752 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:33.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:33.805 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:33.802+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:33.885 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:33.886 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:33.890 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:33.888+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:33.894 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:33.990 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:33.989+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:34.061 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:34.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 27/21 ECSubWrite(tid=74, reqid=client.4418.0:190, at_version=27'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.807457393, 2021-01-15T19:08:34.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002, 2021-01-15T19:08:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251833+0000", 2021-01-15T19:08:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8488000000000001e-05 2021-01-15T19:08:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251871+0000", 2021-01-15T19:08:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0378e-05 2021-01-15T19:08:34.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251902+0000", 2021-01-15T19:08:34.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3121e-05 2021-01-15T19:08:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251915+0000", 2021-01-15T19:08:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3499999999999996e-06 2021-01-15T19:08:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251920+0000", 2021-01-15T19:08:34.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108286 2021-01-15T19:08:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252028+0000", 2021-01-15T19:08:34.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1652e-05 2021-01-15T19:08:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252040+0000", 2021-01-15T19:08:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002924991 2021-01-15T19:08:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.254965+0000", 2021-01-15T19:08:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4570000000000001e-05 2021-01-15T19:08:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255020+0000", 2021-01-15T19:08:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031868360000000002 2021-01-15T19:08:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 28/21 ECSubWrite(tid=45, reqid=client.4418.0:204, at_version=28'16, trim_to=0'0, roll_forward_to=26'14))", 2021-01-15T19:08:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.803843007000001, 2021-01-15T19:08:34.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002, 2021-01-15T19:08:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255447+0000", 2021-01-15T19:08:34.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1822999999999997e-05 2021-01-15T19:08:34.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255479+0000", 2021-01-15T19:08:34.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7513999999999999e-05 2021-01-15T19:08:34.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255497+0000", 2021-01-15T19:08:34.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3723000000000001e-05 2021-01-15T19:08:34.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255510+0000", 2021-01-15T19:08:34.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0780000000000003e-06 2021-01-15T19:08:34.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255515+0000", 2021-01-15T19:08:34.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2679999999999998e-05 2021-01-15T19:08:34.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255558+0000", 2021-01-15T19:08:34.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.695e-06 2021-01-15T19:08:34.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.255566+0000", 2021-01-15T19:08:34.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028394660000000001 2021-01-15T19:08:34.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258405+0000", 2021-01-15T19:08:34.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9791999999999999e-05 2021-01-15T19:08:34.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:16.258435+0000", 2021-01-15T19:08:34.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029877710000000002 2021-01-15T19:08:34.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.088 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.089 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 36/21 ECSubWrite(tid=30, reqid=client.4418.0:244, at_version=36'12, trim_to=0'0, roll_forward_to=25'10))", 2021-01-15T19:08:34.089 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:34.089 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7664592450000001, 2021-01-15T19:08:34.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034199159999999998, 2021-01-15T19:08:34.089 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.090 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.090 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:34.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292831+0000", 2021-01-15T19:08:34.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5223999999999999e-05 2021-01-15T19:08:34.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292876+0000", 2021-01-15T19:08:34.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.614e-05 2021-01-15T19:08:34.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292902+0000", 2021-01-15T19:08:34.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2162e-05 2021-01-15T19:08:34.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292915+0000", 2021-01-15T19:08:34.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2430000000000005e-06 2021-01-15T19:08:34.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292923+0000", 2021-01-15T19:08:34.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4569e-05 2021-01-15T19:08:34.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292967+0000", 2021-01-15T19:08:34.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2889999999999998e-06 2021-01-15T19:08:34.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292976+0000", 2021-01-15T19:08:34.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032267860000000002 2021-01-15T19:08:34.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296202+0000", 2021-01-15T19:08:34.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8503e-05 2021-01-15T19:08:34.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296251+0000", 2021-01-15T19:08:34.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034199159999999998 2021-01-15T19:08:34.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/21 ECSubWrite(tid=71, reqid=client.4418.0:252, at_version=36'23, trim_to=0'0, roll_forward_to=34'21))", 2021-01-15T19:08:34.102 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:34.102 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7477573460000002, 2021-01-15T19:08:34.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003352748, 2021-01-15T19:08:34.103 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.103 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.103 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:34.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.311533+0000", 2021-01-15T19:08:34.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047739199999999999 2021-01-15T19:08:34.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312010+0000", 2021-01-15T19:08:34.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1661000000000001e-05 2021-01-15T19:08:34.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312032+0000", 2021-01-15T19:08:34.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1812e-05 2021-01-15T19:08:34.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312044+0000", 2021-01-15T19:08:34.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.038e-06 2021-01-15T19:08:34.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312052+0000", 2021-01-15T19:08:34.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3818999999999993e-05 2021-01-15T19:08:34.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312116+0000", 2021-01-15T19:08:34.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4571e-05 2021-01-15T19:08:34.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.312130+0000", 2021-01-15T19:08:34.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002722431 2021-01-15T19:08:34.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314853+0000", 2021-01-15T19:08:34.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3024000000000001e-05 2021-01-15T19:08:34.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.314886+0000", 2021-01-15T19:08:34.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003352748 2021-01-15T19:08:34.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.114 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.115 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/21 ECSubWrite(tid=32, reqid=osd.5.0:32, at_version=37'14, trim_to=0'0, roll_forward_to=36'12))", 2021-01-15T19:08:34.115 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:25.282532+0000", 2021-01-15T19:08:34.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.7767580330000001, 2021-01-15T19:08:34.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029664169999999998, 2021-01-15T19:08:34.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.116 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.116 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282532+0000", 2021-01-15T19:08:34.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282532+0000", 2021-01-15T19:08:34.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6085999999999998e-05 2021-01-15T19:08:34.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282588+0000", 2021-01-15T19:08:34.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0889999999999997e-05 2021-01-15T19:08:34.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282619+0000", 2021-01-15T19:08:34.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7946999999999999e-05 2021-01-15T19:08:34.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282637+0000", 2021-01-15T19:08:34.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4449999999999996e-06 2021-01-15T19:08:34.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282646+0000", 2021-01-15T19:08:34.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2564000000000001e-05 2021-01-15T19:08:34.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282688+0000", 2021-01-15T19:08:34.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.85e-06 2021-01-15T19:08:34.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.282697+0000", 2021-01-15T19:08:34.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027569029999999998 2021-01-15T19:08:34.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.285454+0000", 2021-01-15T19:08:34.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4731999999999998e-05 2021-01-15T19:08:34.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:25.285499+0000", 2021-01-15T19:08:34.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029664169999999998 2021-01-15T19:08:34.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.126 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=84, reqid=client.4418.0:311, at_version=41'12, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:34.127 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:34.127 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8630297599999999, 2021-01-15T19:08:34.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049070149999999998, 2021-01-15T19:08:34.128 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.128 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:34.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:34.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065051599999999996 2021-01-15T19:08:34.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196911+0000", 2021-01-15T19:08:34.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2234000000000001e-05 2021-01-15T19:08:34.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196933+0000", 2021-01-15T19:08:34.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4019000000000001e-05 2021-01-15T19:08:34.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196947+0000", 2021-01-15T19:08:34.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1910000000000003e-06 2021-01-15T19:08:34.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196954+0000", 2021-01-15T19:08:34.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7198999999999999e-05 2021-01-15T19:08:34.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197012+0000", 2021-01-15T19:08:34.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5761999999999999e-05 2021-01-15T19:08:34.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197027+0000", 2021-01-15T19:08:34.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004104697 2021-01-15T19:08:34.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.201132+0000", 2021-01-15T19:08:34.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5397e-05 2021-01-15T19:08:34.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.201167+0000", 2021-01-15T19:08:34.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049070149999999998 2021-01-15T19:08:34.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.139 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=87, reqid=client.4418.0:313, at_version=41'14, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:34.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:34.140 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8546794320000002, 2021-01-15T19:08:34.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998, 2021-01-15T19:08:34.140 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.140 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:34.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:34.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0902000000000002e-05 2021-01-15T19:08:34.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204662+0000", 2021-01-15T19:08:34.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8193e-05 2021-01-15T19:08:34.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204680+0000", 2021-01-15T19:08:34.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958e-05 2021-01-15T19:08:34.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204698+0000", 2021-01-15T19:08:34.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2021-01-15T19:08:34.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204706+0000", 2021-01-15T19:08:34.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2994000000000002e-05 2021-01-15T19:08:34.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204749+0000", 2021-01-15T19:08:34.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491999999999999e-05 2021-01-15T19:08:34.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204760+0000", 2021-01-15T19:08:34.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056169770000000004 2021-01-15T19:08:34.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210377+0000", 2021-01-15T19:08:34.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8433999999999997e-05 2021-01-15T19:08:34.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210425+0000", 2021-01-15T19:08:34.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998 2021-01-15T19:08:34.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.151 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.152 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=91, reqid=client.4418.0:313, at_version=0'0, trim_to=0'0, roll_forward_to=41'14))", 2021-01-15T19:08:34.152 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.211528+0000", 2021-01-15T19:08:34.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.847761942, 2021-01-15T19:08:34.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003814684, 2021-01-15T19:08:34.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.154 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211528+0000", 2021-01-15T19:08:34.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211528+0000", 2021-01-15T19:08:34.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3132000000000005e-05 2021-01-15T19:08:34.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211591+0000", 2021-01-15T19:08:34.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6620999999999999e-05 2021-01-15T19:08:34.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211608+0000", 2021-01-15T19:08:34.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8450000000000001e-05 2021-01-15T19:08:34.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211627+0000", 2021-01-15T19:08:34.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1795e-05 2021-01-15T19:08:34.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211638+0000", 2021-01-15T19:08:34.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3019000000000002e-05 2021-01-15T19:08:34.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211691+0000", 2021-01-15T19:08:34.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3823e-05 2021-01-15T19:08:34.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.211705+0000", 2021-01-15T19:08:34.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003581135 2021-01-15T19:08:34.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.215286+0000", 2021-01-15T19:08:34.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6709000000000002e-05 2021-01-15T19:08:34.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.215343+0000", 2021-01-15T19:08:34.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003814684 2021-01-15T19:08:34.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.165 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.166 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=115, reqid=client.4418.0:324, at_version=41'23, trim_to=0'0, roll_forward_to=41'21))", 2021-01-15T19:08:34.166 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:34.166 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8470595730000001, 2021-01-15T19:08:34.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00487003, 2021-01-15T19:08:34.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.167 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:34.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3484000000000002e-05 2021-01-15T19:08:34.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212314+0000", 2021-01-15T19:08:34.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7509000000000001e-05 2021-01-15T19:08:34.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212332+0000", 2021-01-15T19:08:34.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1646e-05 2021-01-15T19:08:34.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212343+0000", 2021-01-15T19:08:34.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:08:34.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212349+0000", 2021-01-15T19:08:34.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3396999999999997e-05 2021-01-15T19:08:34.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212402+0000", 2021-01-15T19:08:34.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3617e-05 2021-01-15T19:08:34.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212416+0000", 2021-01-15T19:08:34.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046101270000000003 2021-01-15T19:08:34.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217026+0000", 2021-01-15T19:08:34.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4484e-05 2021-01-15T19:08:34.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217101+0000", 2021-01-15T19:08:34.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00487003 2021-01-15T19:08:34.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.178 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.178 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.179 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=117, reqid=client.4418.0:326, at_version=41'25, trim_to=0'0, roll_forward_to=41'21))", 2021-01-15T19:08:34.179 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217776+0000", 2021-01-15T19:08:34.179 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8415139399999996, 2021-01-15T19:08:34.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033786580000000001, 2021-01-15T19:08:34.179 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.180 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217776+0000", 2021-01-15T19:08:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217776+0000", 2021-01-15T19:08:34.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014095899999999999 2021-01-15T19:08:34.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217917+0000", 2021-01-15T19:08:34.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2052000000000002e-05 2021-01-15T19:08:34.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217939+0000", 2021-01-15T19:08:34.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-05 2021-01-15T19:08:34.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217958+0000", 2021-01-15T19:08:34.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7789999999999998e-06 2021-01-15T19:08:34.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217964+0000", 2021-01-15T19:08:34.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6045999999999999e-05 2021-01-15T19:08:34.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218011+0000", 2021-01-15T19:08:34.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2300000000000001e-05 2021-01-15T19:08:34.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218023+0000", 2021-01-15T19:08:34.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031022229999999999 2021-01-15T19:08:34.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221125+0000", 2021-01-15T19:08:34.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9929000000000001e-05 2021-01-15T19:08:34.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221155+0000", 2021-01-15T19:08:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033786580000000001 2021-01-15T19:08:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.191 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=42, reqid=client.4418.0:332, at_version=41'23, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:34.191 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.221646+0000", 2021-01-15T19:08:34.191 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8376447130000004, 2021-01-15T19:08:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031358509999999998, 2021-01-15T19:08:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221646+0000", 2021-01-15T19:08:34.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221646+0000", 2021-01-15T19:08:34.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010761500000000001 2021-01-15T19:08:34.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221753+0000", 2021-01-15T19:08:34.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4093e-05 2021-01-15T19:08:34.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221767+0000", 2021-01-15T19:08:34.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7309999999999999e-06 2021-01-15T19:08:34.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221776+0000", 2021-01-15T19:08:34.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3340000000000003e-06 2021-01-15T19:08:34.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221780+0000", 2021-01-15T19:08:34.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4490000000000003e-05 2021-01-15T19:08:34.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221825+0000", 2021-01-15T19:08:34.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2089999999999999e-06 2021-01-15T19:08:34.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221832+0000", 2021-01-15T19:08:34.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002886422 2021-01-15T19:08:34.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224718+0000", 2021-01-15T19:08:34.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2957e-05 2021-01-15T19:08:34.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224781+0000", 2021-01-15T19:08:34.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031358509999999998 2021-01-15T19:08:34.203 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.203 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.204 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=119, reqid=client.4418.0:327, at_version=0'0, trim_to=0'0, roll_forward_to=41'26))", 2021-01-15T19:08:34.204 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222670+0000", 2021-01-15T19:08:34.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8366198450000004, 2021-01-15T19:08:34.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039183689999999997, 2021-01-15T19:08:34.205 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.205 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.205 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222670+0000", 2021-01-15T19:08:34.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222670+0000", 2021-01-15T19:08:34.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4183e-05 2021-01-15T19:08:34.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222715+0000", 2021-01-15T19:08:34.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3796e-05 2021-01-15T19:08:34.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222728+0000", 2021-01-15T19:08:34.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7295999999999998e-05 2021-01-15T19:08:34.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222746+0000", 2021-01-15T19:08:34.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1580000000000002e-06 2021-01-15T19:08:34.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222752+0000", 2021-01-15T19:08:34.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4154999999999997e-05 2021-01-15T19:08:34.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222826+0000", 2021-01-15T19:08:34.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3285000000000001e-05 2021-01-15T19:08:34.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222839+0000", 2021-01-15T19:08:34.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037127660000000002 2021-01-15T19:08:34.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226552+0000", 2021-01-15T19:08:34.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6730000000000002e-05 2021-01-15T19:08:34.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226589+0000", 2021-01-15T19:08:34.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039183689999999997 2021-01-15T19:08:34.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.216 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.217 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:34.217 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:34.218 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8356910070000003, 2021-01-15T19:08:34.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:34.218 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.218 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.218 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:34.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:34.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:34.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:34.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:34.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:34.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:34.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:34.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:34.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:34.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:34.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:34.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:34.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:34.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:34.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:34.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:34.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:34.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:34.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.229 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.230 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=44, reqid=client.4418.0:334, at_version=41'25, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:34.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:34.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8339967899999996, 2021-01-15T19:08:34.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001, 2021-01-15T19:08:34.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:34.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:34.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9133000000000004e-05 2021-01-15T19:08:34.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225363+0000", 2021-01-15T19:08:34.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2021-01-15T19:08:34.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225376+0000", 2021-01-15T19:08:34.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0949e-05 2021-01-15T19:08:34.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225387+0000", 2021-01-15T19:08:34.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7710000000000001e-06 2021-01-15T19:08:34.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225394+0000", 2021-01-15T19:08:34.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5732999999999997e-05 2021-01-15T19:08:34.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225440+0000", 2021-01-15T19:08:34.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5984000000000001e-05 2021-01-15T19:08:34.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225456+0000", 2021-01-15T19:08:34.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059787429999999999 2021-01-15T19:08:34.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231434+0000", 2021-01-15T19:08:34.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2653000000000001e-05 2021-01-15T19:08:34.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231457+0000", 2021-01-15T19:08:34.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001 2021-01-15T19:08:34.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.242 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.242 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.242 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=45, reqid=client.4418.0:335, at_version=41'26, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:34.243 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:34.243 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8323100639999996, 2021-01-15T19:08:34.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047473949999999997, 2021-01-15T19:08:34.243 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.243 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:34.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:34.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0362000000000003e-05 2021-01-15T19:08:34.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227031+0000", 2021-01-15T19:08:34.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9890000000000001e-05 2021-01-15T19:08:34.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227050+0000", 2021-01-15T19:08:34.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7663e-05 2021-01-15T19:08:34.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227068+0000", 2021-01-15T19:08:34.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4250000000000003e-06 2021-01-15T19:08:34.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227075+0000", 2021-01-15T19:08:34.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8297999999999997e-05 2021-01-15T19:08:34.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227123+0000", 2021-01-15T19:08:34.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0387000000000001e-05 2021-01-15T19:08:34.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227133+0000", 2021-01-15T19:08:34.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004554598 2021-01-15T19:08:34.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231688+0000", 2021-01-15T19:08:34.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9771999999999998e-05 2021-01-15T19:08:34.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231728+0000", 2021-01-15T19:08:34.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047473949999999997 2021-01-15T19:08:34.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.255 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.256 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:338 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:34.256 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179693+0000", 2021-01-15T19:08:34.256 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.879596888, 2021-01-15T19:08:34.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048277429999999998, 2021-01-15T19:08:34.256 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.257 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:34.257 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:34.257 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:34.257 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 338 2021-01-15T19:08:34.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179693+0000", 2021-01-15T19:08:34.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179693+0000", 2021-01-15T19:08:34.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5828999999999999e-05 2021-01-15T19:08:34.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179729+0000", 2021-01-15T19:08:34.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0650000000000001e-06 2021-01-15T19:08:34.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179736+0000", 2021-01-15T19:08:34.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2021-01-15T19:08:34.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179758+0000", 2021-01-15T19:08:34.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2945000000000001e-05 2021-01-15T19:08:34.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179771+0000", 2021-01-15T19:08:34.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4350000000000004e-05 2021-01-15T19:08:34.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179855+0000", 2021-01-15T19:08:34.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3669000000000001e-05 2021-01-15T19:08:34.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:34.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179869+0000", 2021-01-15T19:08:34.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019447589999999999 2021-01-15T19:08:34.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.181814+0000", 2021-01-15T19:08:34.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015279099999999999 2021-01-15T19:08:34.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:34.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.181966+0000", 2021-01-15T19:08:34.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025547259999999998 2021-01-15T19:08:34.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.184521+0000", 2021-01-15T19:08:34.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048277429999999998 2021-01-15T19:08:34.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.272 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.273 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:339 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:34.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179837+0000", 2021-01-15T19:08:34.273 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8794533549999999, 2021-01-15T19:08:34.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052755500000000004, 2021-01-15T19:08:34.273 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:34.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:34.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:34.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 339 2021-01-15T19:08:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179837+0000", 2021-01-15T19:08:34.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179837+0000", 2021-01-15T19:08:34.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6734999999999998e-05 2021-01-15T19:08:34.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179894+0000", 2021-01-15T19:08:34.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8279999999999998e-06 2021-01-15T19:08:34.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179903+0000", 2021-01-15T19:08:34.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4875000000000001e-05 2021-01-15T19:08:34.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179918+0000", 2021-01-15T19:08:34.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6220000000000003e-06 2021-01-15T19:08:34.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179925+0000", 2021-01-15T19:08:34.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2964000000000003e-05 2021-01-15T19:08:34.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179978+0000", 2021-01-15T19:08:34.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3149999999999998e-06 2021-01-15T19:08:34.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:34.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179987+0000", 2021-01-15T19:08:34.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029067009999999998 2021-01-15T19:08:34.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.182894+0000", 2021-01-15T19:08:34.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8940000000000001e-05 2021-01-15T19:08:34.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:34.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.182963+0000", 2021-01-15T19:08:34.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00214957 2021-01-15T19:08:34.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.185112+0000", 2021-01-15T19:08:34.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052755500000000004 2021-01-15T19:08:34.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.288 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:340 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:34.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:34.289 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8793165109999999, 2021-01-15T19:08:34.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056637780000000004, 2021-01-15T19:08:34.290 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.290 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:34.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:34.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:34.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.291 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 340 2021-01-15T19:08:34.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.291 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:34.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:34.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6628000000000001e-05 2021-01-15T19:08:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180030+0000", 2021-01-15T19:08:34.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:08:34.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180038+0000", 2021-01-15T19:08:34.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3541e-05 2021-01-15T19:08:34.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180051+0000", 2021-01-15T19:08:34.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:08:34.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180056+0000", 2021-01-15T19:08:34.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3109999999999998e-05 2021-01-15T19:08:34.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180110+0000", 2021-01-15T19:08:34.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9489999999999996e-06 2021-01-15T19:08:34.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:34.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180118+0000", 2021-01-15T19:08:34.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033250100000000002 2021-01-15T19:08:34.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.183443+0000", 2021-01-15T19:08:34.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2630000000000002e-05 2021-01-15T19:08:34.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:34.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.183505+0000", 2021-01-15T19:08:34.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021323370000000002 2021-01-15T19:08:34.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.185638+0000", 2021-01-15T19:08:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056637780000000004 2021-01-15T19:08:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 45/21 ECSubWrite(tid=54, reqid=client.4418.0:370, at_version=45'33, trim_to=0'0, roll_forward_to=44'31))", 2021-01-15T19:08:34.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.202625+0000", 2021-01-15T19:08:34.306 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.85666480300000003, 2021-01-15T19:08:34.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036218729999999998, 2021-01-15T19:08:34.306 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.306 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.307 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202625+0000", 2021-01-15T19:08:34.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202625+0000", 2021-01-15T19:08:34.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016847499999999999 2021-01-15T19:08:34.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202794+0000", 2021-01-15T19:08:34.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7628e-05 2021-01-15T19:08:34.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202822+0000", 2021-01-15T19:08:34.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5092000000000001e-05 2021-01-15T19:08:34.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202837+0000", 2021-01-15T19:08:34.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.238e-06 2021-01-15T19:08:34.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202844+0000", 2021-01-15T19:08:34.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8248000000000001e-05 2021-01-15T19:08:34.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202912+0000", 2021-01-15T19:08:34.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2921999999999999e-05 2021-01-15T19:08:34.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202925+0000", 2021-01-15T19:08:34.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032733829999999999 2021-01-15T19:08:34.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206198+0000", 2021-01-15T19:08:34.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8887000000000001e-05 2021-01-15T19:08:34.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206247+0000", 2021-01-15T19:08:34.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036218729999999998 2021-01-15T19:08:34.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.319 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 45/21 ECSubWrite(tid=55, reqid=client.4418.0:372, at_version=45'35, trim_to=0'0, roll_forward_to=44'31))", 2021-01-15T19:08:34.319 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.206009+0000", 2021-01-15T19:08:34.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.85328081700000002, 2021-01-15T19:08:34.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048428999999999998, 2021-01-15T19:08:34.320 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:34.320 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:34.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:34.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:34.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206009+0000", 2021-01-15T19:08:34.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:34.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:34.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206009+0000", 2021-01-15T19:08:34.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197584 2021-01-15T19:08:34.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:34.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206207+0000", 2021-01-15T19:08:34.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7834000000000002e-05 2021-01-15T19:08:34.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:34.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206235+0000", 2021-01-15T19:08:34.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059999999999999e-05 2021-01-15T19:08:34.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:34.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206252+0000", 2021-01-15T19:08:34.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0770000000000002e-06 2021-01-15T19:08:34.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206259+0000", 2021-01-15T19:08:34.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6487999999999999e-05 2021-01-15T19:08:34.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:34.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206305+0000", 2021-01-15T19:08:34.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2275e-05 2021-01-15T19:08:34.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206318+0000", 2021-01-15T19:08:34.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044787849999999999 2021-01-15T19:08:34.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.210797+0000", 2021-01-15T19:08:34.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5797000000000001e-05 2021-01-15T19:08:34.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:34.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:34.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:34.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.210852+0000", 2021-01-15T19:08:34.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048428999999999998 2021-01-15T19:08:34.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:34.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:34.333 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:34.335 INFO:tasks.rados.rados.0.smithi001.stdout:140: done (5 left) 2021-01-15T19:08:34.335 INFO:tasks.rados.rados.0.smithi001.stdout:141: done (4 left) 2021-01-15T19:08:34.336 INFO:tasks.rados.rados.0.smithi001.stdout:142: done (3 left) 2021-01-15T19:08:34.336 INFO:tasks.rados.rados.0.smithi001.stdout:143: done (2 left) 2021-01-15T19:08:34.336 INFO:tasks.rados.rados.0.smithi001.stdout:144: done (1 left) 2021-01-15T19:08:34.336 INFO:tasks.rados.rados.0.smithi001.stdout:145: done (0 left) 2021-01-15T19:08:34.337 INFO:tasks.rados.rados.0.smithi001.stdout:146: snap_remove snap 12 2021-01-15T19:08:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:08:34.337 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:34.333+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:34.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:34.435 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:34.434+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:34.520 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:34.520 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:34.520 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:34.521 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:34.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:34.537 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:34.535+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:34.639 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:34.637+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:34.664 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:34.664 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:34.665 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:34.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:34.739 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:34.737+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:34.840 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:34.839+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:34.847 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:08:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:154 3.fs0 3:f27a3046:::smithi00140323-1:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.607286834, 2021-01-15T19:08:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002, 2021-01-15T19:08:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 154 2021-01-15T19:08:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4352999999999999e-05 2021-01-15T19:08:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237631+0000", 2021-01-15T19:08:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3304e-05 2021-01-15T19:08:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237644+0000", 2021-01-15T19:08:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1736e-05 2021-01-15T19:08:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237666+0000", 2021-01-15T19:08:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0607e-05 2021-01-15T19:08:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237676+0000", 2021-01-15T19:08:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3862e-05 2021-01-15T19:08:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237730+0000", 2021-01-15T19:08:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1849999999999998e-06 2021-01-15T19:08:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237737+0000", 2021-01-15T19:08:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014276250000000001 2021-01-15T19:08:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239165+0000", 2021-01-15T19:08:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7447000000000007e-05 2021-01-15T19:08:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239232+0000", 2021-01-15T19:08:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061349480000000003 2021-01-15T19:08:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245367+0000", 2021-01-15T19:08:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095382000000000002 2021-01-15T19:08:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254906+0000", 2021-01-15T19:08:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113465 2021-01-15T19:08:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255019+0000", 2021-01-15T19:08:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000454659 2021-01-15T19:08:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255474+0000", 2021-01-15T19:08:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002 2021-01-15T19:08:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:155 3.fs0 3:f27a3046:::smithi00140323-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.607125726, 2021-01-15T19:08:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999, 2021-01-15T19:08:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 155 2021-01-15T19:08:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9607999999999998e-05 2021-01-15T19:08:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237767+0000", 2021-01-15T19:08:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3810000000000001e-06 2021-01-15T19:08:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237771+0000", 2021-01-15T19:08:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5639999999999999e-06 2021-01-15T19:08:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237779+0000", 2021-01-15T19:08:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.348e-06 2021-01-15T19:08:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237782+0000", 2021-01-15T19:08:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3377000000000002e-05 2021-01-15T19:08:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237816+0000", 2021-01-15T19:08:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000003e-06 2021-01-15T19:08:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237822+0000", 2021-01-15T19:08:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552276 2021-01-15T19:08:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239374+0000", 2021-01-15T19:08:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239415+0000", 2021-01-15T19:08:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010934268 2021-01-15T19:08:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250349+0000", 2021-01-15T19:08:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011671 2021-01-15T19:08:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250466+0000", 2021-01-15T19:08:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050409820000000003 2021-01-15T19:08:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255507+0000", 2021-01-15T19:08:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0531000000000005e-05 2021-01-15T19:08:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255587+0000", 2021-01-15T19:08:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122043 2021-01-15T19:08:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255709+0000", 2021-01-15T19:08:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999 2021-01-15T19:08:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:156 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~442368] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.603285367000002, 2021-01-15T19:08:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759, 2021-01-15T19:08:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156 2021-01-15T19:08:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030551200000000003 2021-01-15T19:08:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241873+0000", 2021-01-15T19:08:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6080000000000002e-06 2021-01-15T19:08:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241883+0000", 2021-01-15T19:08:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241898+0000", 2021-01-15T19:08:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4010000000000002e-06 2021-01-15T19:08:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241904+0000", 2021-01-15T19:08:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6971999999999997e-05 2021-01-15T19:08:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241951+0000", 2021-01-15T19:08:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104244 2021-01-15T19:08:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242055+0000", 2021-01-15T19:08:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070015099999999999 2021-01-15T19:08:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242755+0000", 2021-01-15T19:08:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0080604460000000006 2021-01-15T19:08:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250816+0000", 2021-01-15T19:08:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00338733 2021-01-15T19:08:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254203+0000", 2021-01-15T19:08:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3444999999999998e-05 2021-01-15T19:08:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254287+0000", 2021-01-15T19:08:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759 2021-01-15T19:08:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:157 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~401408] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.600479140000001, 2021-01-15T19:08:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335, 2021-01-15T19:08:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 157 2021-01-15T19:08:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000301899 2021-01-15T19:08:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244676+0000", 2021-01-15T19:08:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6430000000000001e-06 2021-01-15T19:08:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244685+0000", 2021-01-15T19:08:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5293000000000001e-05 2021-01-15T19:08:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244700+0000", 2021-01-15T19:08:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6499999999999996e-06 2021-01-15T19:08:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244708+0000", 2021-01-15T19:08:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1064999999999997e-05 2021-01-15T19:08:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244789+0000", 2021-01-15T19:08:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117805 2021-01-15T19:08:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244906+0000", 2021-01-15T19:08:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070339100000000004 2021-01-15T19:08:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245610+0000", 2021-01-15T19:08:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087607599999999994 2021-01-15T19:08:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107695 2021-01-15T19:08:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254478+0000", 2021-01-15T19:08:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109134 2021-01-15T19:08:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254587+0000", 2021-01-15T19:08:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335 2021-01-15T19:08:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:158 3.cs0 3:3ef45b66:::smithi00140323-11:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.600093123000001, 2021-01-15T19:08:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441, 2021-01-15T19:08:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 158 2021-01-15T19:08:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5699999999999999e-05 2021-01-15T19:08:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244816+0000", 2021-01-15T19:08:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6839999999999996e-06 2021-01-15T19:08:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244823+0000", 2021-01-15T19:08:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4626e-05 2021-01-15T19:08:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244838+0000", 2021-01-15T19:08:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6500000000000001e-06 2021-01-15T19:08:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244844+0000", 2021-01-15T19:08:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004205149 2021-01-15T19:08:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249049+0000", 2021-01-15T19:08:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015840599999999999 2021-01-15T19:08:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249207+0000", 2021-01-15T19:08:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071027700000000004 2021-01-15T19:08:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249918+0000", 2021-01-15T19:08:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051411900000000003 2021-01-15T19:08:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255059+0000", 2021-01-15T19:08:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045701699999999999 2021-01-15T19:08:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255516+0000", 2021-01-15T19:08:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8710999999999997e-05 2021-01-15T19:08:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255614+0000", 2021-01-15T19:08:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441 2021-01-15T19:08:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:159 3.cs0 3:3ef45b66:::smithi00140323-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.599960672000002, 2021-01-15T19:08:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195, 2021-01-15T19:08:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 159 2021-01-15T19:08:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0961000000000003e-05 2021-01-15T19:08:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244933+0000", 2021-01-15T19:08:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0480000000000001e-06 2021-01-15T19:08:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244941+0000", 2021-01-15T19:08:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3711e-05 2021-01-15T19:08:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244954+0000", 2021-01-15T19:08:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2390000000000002e-06 2021-01-15T19:08:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244959+0000", 2021-01-15T19:08:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082968859999999998 2021-01-15T19:08:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253256+0000", 2021-01-15T19:08:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000147608 2021-01-15T19:08:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253404+0000", 2021-01-15T19:08:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047189329999999998 2021-01-15T19:08:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258123+0000", 2021-01-15T19:08:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010107000000000001 2021-01-15T19:08:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258224+0000", 2021-01-15T19:08:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019277389999999999 2021-01-15T19:08:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260152+0000", 2021-01-15T19:08:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195 2021-01-15T19:08:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:201 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.596607344999999, 2021-01-15T19:08:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142, 2021-01-15T19:08:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 201 2021-01-15T19:08:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1060000000000002e-05 2021-01-15T19:08:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248307+0000", 2021-01-15T19:08:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6740000000000003e-06 2021-01-15T19:08:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248316+0000", 2021-01-15T19:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8746e-05 2021-01-15T19:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248334+0000", 2021-01-15T19:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2294e-05 2021-01-15T19:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248347+0000", 2021-01-15T19:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6400999999999998e-05 2021-01-15T19:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248393+0000", 2021-01-15T19:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2021-01-15T19:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248400+0000", 2021-01-15T19:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002761197 2021-01-15T19:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251161+0000", 2021-01-15T19:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011450099999999999 2021-01-15T19:08:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251275+0000", 2021-01-15T19:08:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062315829999999997 2021-01-15T19:08:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257507+0000", 2021-01-15T19:08:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142 2021-01-15T19:08:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:202 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.596453969999999, 2021-01-15T19:08:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285, 2021-01-15T19:08:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 202 2021-01-15T19:08:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5448999999999997e-05 2021-01-15T19:08:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248455+0000", 2021-01-15T19:08:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248461+0000", 2021-01-15T19:08:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4307e-05 2021-01-15T19:08:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248476+0000", 2021-01-15T19:08:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5819999999999998e-06 2021-01-15T19:08:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248482+0000", 2021-01-15T19:08:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1987e-05 2021-01-15T19:08:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248564+0000", 2021-01-15T19:08:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5850000000000002e-06 2021-01-15T19:08:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248572+0000", 2021-01-15T19:08:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045194099999999997 2021-01-15T19:08:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253091+0000", 2021-01-15T19:08:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020063399999999999 2021-01-15T19:08:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253292+0000", 2021-01-15T19:08:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079735020000000004 2021-01-15T19:08:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261266+0000", 2021-01-15T19:08:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285 2021-01-15T19:08:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:203 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.596274952000002, 2021-01-15T19:08:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438, 2021-01-15T19:08:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 203 2021-01-15T19:08:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8518999999999999e-05 2021-01-15T19:08:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248607+0000", 2021-01-15T19:08:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8810000000000004e-06 2021-01-15T19:08:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248612+0000", 2021-01-15T19:08:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5719999999999997e-06 2021-01-15T19:08:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248619+0000", 2021-01-15T19:08:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5899999999999999e-06 2021-01-15T19:08:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248623+0000", 2021-01-15T19:08:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5608000000000001e-05 2021-01-15T19:08:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248668+0000", 2021-01-15T19:08:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0166000000000001e-05 2021-01-15T19:08:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248679+0000", 2021-01-15T19:08:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048830590000000004 2021-01-15T19:08:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253562+0000", 2021-01-15T19:08:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7e-05 2021-01-15T19:08:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253659+0000", 2021-01-15T19:08:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081539850000000007 2021-01-15T19:08:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261813+0000", 2021-01-15T19:08:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438 2021-01-15T19:08:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:204 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat,delete] snapc 3=[3] ondisk+write+known_if_redirected e28)", 2021-01-15T19:08:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.596190745000001, 2021-01-15T19:08:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532, 2021-01-15T19:08:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 204 2021-01-15T19:08:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8349000000000002e-05 2021-01-15T19:08:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248701+0000", 2021-01-15T19:08:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0620000000000001e-06 2021-01-15T19:08:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248709+0000", 2021-01-15T19:08:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7700000000000001e-06 2021-01-15T19:08:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248717+0000", 2021-01-15T19:08:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5920000000000001e-06 2021-01-15T19:08:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248720+0000", 2021-01-15T19:08:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0898e-05 2021-01-15T19:08:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248761+0000", 2021-01-15T19:08:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-06 2021-01-15T19:08:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248770+0000", 2021-01-15T19:08:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052185720000000003 2021-01-15T19:08:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253989+0000", 2021-01-15T19:08:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105809 2021-01-15T19:08:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254095+0000", 2021-01-15T19:08:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078837199999999997 2021-01-15T19:08:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254883+0000", 2021-01-15T19:08:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083722769999999992 2021-01-15T19:08:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263255+0000", 2021-01-15T19:08:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111839 2021-01-15T19:08:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263367+0000", 2021-01-15T19:08:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015969899999999999 2021-01-15T19:08:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263527+0000", 2021-01-15T19:08:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532 2021-01-15T19:08:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:245 3.fs0 3:fae92f17:::smithi00140323-19:head [create,setxattr YOk\\Qsqnjk`Uh[icZasWhcZnad[h[k[Qmk]TihbR (1823) in=40b,setxattr QnfshqY (34) in=7b,setxattr V_tZOrZWhpPekPNs_toQaraRlW\\sWffpapfPROXb (1614) in=40b,setxattr aiTbV]hXcpXjkS]VQonZZkfogbUSnO[bajbZWdRf (839) in=40b,setxattr pZdlecZrV]qp]`sTkf_bSjZVZjhbScXfb[Sr_bsa (18524) in=40b,setxattr kXtOdjofgYOa_Tck\\nYsbVOfgtldphnQ`]P[j]lX (1295) in=40b,setxattr dRpscRhc^OtlbThWSmXVSNge]s (34) in=26b,setxattr T\\s[QdNhlRZh\\\\[X]NpSccmfRcqRaXScZOZ\\ZR^e (1999) in=40b,setxattr YTcfnqlbrRa^rhctrncU_Wa]laamPmYsaqg_f^N` (1471) in=40b,setxattr l_Qeh\\RcafrRYN_YZf_iiaWagZfe[sop]^\\qTggq (561) in=40b,setxattr NShVOQe\\dRYY[tPettNbkgkTVonNshnbYeePWcla (314) in=40b,setxattr ^iVidVQX]Yn`XTY\\pOXm`_PNX`qeckhbQoR`aQWZ (8841) in=40b,setxattr _jkmO]cXrN_S^act\\QOOjpUZOj_Qf^otnZbpWZPU (981) in=40b,setxattr bNbhPX[YP[lppmc`PkWcpnS`XsQNpW\\UYnWr\\_RX (1927) in=40b,setxattr talTebhpYS[[TXcgodoQR\\tf`fVekppjlkWnTqf[ (1753) in=40b,setxattr rp\\XTgd\\X^qaodkhY[elq^RagRhN_NR^j^NmjOUh (806) in=40b,setxattr ]\\oXSo`PWafUO^dbhR^i`hifQgWjRhR[rRa^bnej (70) in=40b,setxattr UkYrdYaZakpnReakOdc\\^bO[YPqfdkdtbNkghaSQ (279) in=40b,setxattr YtUs_hcRdQqgZV]NaadOflcb\\_c^\\Zg_Y\\]snXXZ (701) in=40b,setxattr NRZdh]WclahTe^_btdqe\\^mVYqQOaXaa\\aPVXWnk (19016) in=40b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.544356504, 2021-01-15T19:08:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083409910000000007, 2021-01-15T19:08:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 245 2021-01-15T19:08:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300497+0000", 2021-01-15T19:08:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122779 2021-01-15T19:08:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300619+0000", 2021-01-15T19:08:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0987e-05 2021-01-15T19:08:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300630+0000", 2021-01-15T19:08:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0562000000000001e-05 2021-01-15T19:08:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300651+0000", 2021-01-15T19:08:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329999999999999e-05 2021-01-15T19:08:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300661+0000", 2021-01-15T19:08:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6162000000000002e-05 2021-01-15T19:08:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300718+0000", 2021-01-15T19:08:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105159 2021-01-15T19:08:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300823+0000", 2021-01-15T19:08:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011057879999999999 2021-01-15T19:08:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301928+0000", 2021-01-15T19:08:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064543059999999999 2021-01-15T19:08:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308383+0000", 2021-01-15T19:08:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031878700000000001 2021-01-15T19:08:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308702+0000", 2021-01-15T19:08:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000136131 2021-01-15T19:08:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308838+0000", 2021-01-15T19:08:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083409910000000007 2021-01-15T19:08:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:246 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.544141933000001, 2021-01-15T19:08:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005, 2021-01-15T19:08:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 246 2021-01-15T19:08:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1804000000000001e-05 2021-01-15T19:08:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300763+0000", 2021-01-15T19:08:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5610000000000004e-06 2021-01-15T19:08:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300770+0000", 2021-01-15T19:08:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300782+0000", 2021-01-15T19:08:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4460000000000003e-06 2021-01-15T19:08:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300786+0000", 2021-01-15T19:08:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003399596 2021-01-15T19:08:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304186+0000", 2021-01-15T19:08:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125399 2021-01-15T19:08:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304311+0000", 2021-01-15T19:08:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057843119999999998 2021-01-15T19:08:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310095+0000", 2021-01-15T19:08:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005 2021-01-15T19:08:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:247 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.544024820000001, 2021-01-15T19:08:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281, 2021-01-15T19:08:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 247 2021-01-15T19:08:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2098999999999997e-05 2021-01-15T19:08:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300880+0000", 2021-01-15T19:08:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2021-01-15T19:08:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300887+0000", 2021-01-15T19:08:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.152e-05 2021-01-15T19:08:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300898+0000", 2021-01-15T19:08:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.904e-06 2021-01-15T19:08:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300903+0000", 2021-01-15T19:08:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039909860000000002 2021-01-15T19:08:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304894+0000", 2021-01-15T19:08:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012664060000000001 2021-01-15T19:08:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306161+0000", 2021-01-15T19:08:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044999549999999999 2021-01-15T19:08:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310661+0000", 2021-01-15T19:08:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281 2021-01-15T19:08:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:248 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.543910292, 2021-01-15T19:08:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004, 2021-01-15T19:08:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 248 2021-01-15T19:08:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9626999999999998e-05 2021-01-15T19:08:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300993+0000", 2021-01-15T19:08:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1990000000000003e-06 2021-01-15T19:08:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300999+0000", 2021-01-15T19:08:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1737e-05 2021-01-15T19:08:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301010+0000", 2021-01-15T19:08:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.006e-05 2021-01-15T19:08:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301021+0000", 2021-01-15T19:08:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054558510000000003 2021-01-15T19:08:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306476+0000", 2021-01-15T19:08:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000193029 2021-01-15T19:08:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306669+0000", 2021-01-15T19:08:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046625010000000003 2021-01-15T19:08:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.311332+0000", 2021-01-15T19:08:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004 2021-01-15T19:08:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:252 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~647168] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.536171862, 2021-01-15T19:08:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996, 2021-01-15T19:08:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 252 2021-01-15T19:08:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041048799999999998 2021-01-15T19:08:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309092+0000", 2021-01-15T19:08:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:08:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309101+0000", 2021-01-15T19:08:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4919000000000001e-05 2021-01-15T19:08:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309116+0000", 2021-01-15T19:08:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2021-01-15T19:08:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309122+0000", 2021-01-15T19:08:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0148999999999997e-05 2021-01-15T19:08:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309172+0000", 2021-01-15T19:08:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013501799999999999 2021-01-15T19:08:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309307+0000", 2021-01-15T19:08:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015785269999999999 2021-01-15T19:08:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310886+0000", 2021-01-15T19:08:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070233450000000003 2021-01-15T19:08:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.317909+0000", 2021-01-15T19:08:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2075000000000002e-05 2021-01-15T19:08:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318001+0000", 2021-01-15T19:08:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0710999999999994e-05 2021-01-15T19:08:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318072+0000", 2021-01-15T19:08:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996 2021-01-15T19:08:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:285 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~696320] snapc 10=[10] ondisk+write+known_if_redirected e40)", 2021-01-15T19:08:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6578032169999997, 2021-01-15T19:08:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093549089999999998, 2021-01-15T19:08:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 285 2021-01-15T19:08:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041889100000000001 2021-01-15T19:08:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187469+0000", 2021-01-15T19:08:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2860999999999999e-05 2021-01-15T19:08:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187482+0000", 2021-01-15T19:08:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1379e-05 2021-01-15T19:08:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187503+0000", 2021-01-15T19:08:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0691e-05 2021-01-15T19:08:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187514+0000", 2021-01-15T19:08:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9081e-05 2021-01-15T19:08:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187563+0000", 2021-01-15T19:08:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8070000000000002e-06 2021-01-15T19:08:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187570+0000", 2021-01-15T19:08:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019352589999999999 2021-01-15T19:08:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.189505+0000", 2021-01-15T19:08:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011456299999999999 2021-01-15T19:08:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.189620+0000", 2021-01-15T19:08:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000972923 2021-01-15T19:08:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190592+0000", 2021-01-15T19:08:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056078150000000004 2021-01-15T19:08:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196200+0000", 2021-01-15T19:08:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102021 2021-01-15T19:08:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196302+0000", 2021-01-15T19:08:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102618 2021-01-15T19:08:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196405+0000", 2021-01-15T19:08:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093549089999999998 2021-01-15T19:08:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:288 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e40)", 2021-01-15T19:08:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.190414+0000", 2021-01-15T19:08:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6544387949999999, 2021-01-15T19:08:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086577609999999999, 2021-01-15T19:08:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 288 2021-01-15T19:08:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190414+0000", 2021-01-15T19:08:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190414+0000", 2021-01-15T19:08:35.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8944e-05 2021-01-15T19:08:35.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190433+0000", 2021-01-15T19:08:35.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3690000000000007e-06 2021-01-15T19:08:35.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190442+0000", 2021-01-15T19:08:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9399999999999996e-06 2021-01-15T19:08:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190449+0000", 2021-01-15T19:08:35.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.672e-06 2021-01-15T19:08:35.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190451+0000", 2021-01-15T19:08:35.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0055543759999999998 2021-01-15T19:08:35.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196006+0000", 2021-01-15T19:08:35.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111593 2021-01-15T19:08:35.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:35.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196117+0000", 2021-01-15T19:08:35.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001568949 2021-01-15T19:08:35.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.197686+0000", 2021-01-15T19:08:35.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6741000000000002e-05 2021-01-15T19:08:35.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.197773+0000", 2021-01-15T19:08:35.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012991770000000001 2021-01-15T19:08:35.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.199072+0000", 2021-01-15T19:08:35.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086577609999999999 2021-01-15T19:08:35.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:296 3.fs0 3:fae92f17:::smithi00140323-19:head [append 0~729088] snapc 10=[10] ondisk+write+known_if_redirected e40)", 2021-01-15T19:08:35.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.211804+0000", 2021-01-15T19:08:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.6330493779999999, 2021-01-15T19:08:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083155529999999998, 2021-01-15T19:08:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:35.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 296 2021-01-15T19:08:35.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.122 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.211804+0000", 2021-01-15T19:08:35.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.211804+0000", 2021-01-15T19:08:35.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019096400000000001 2021-01-15T19:08:35.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.211995+0000", 2021-01-15T19:08:35.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5130000000000002e-06 2021-01-15T19:08:35.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212001+0000", 2021-01-15T19:08:35.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7623000000000001e-05 2021-01-15T19:08:35.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212019+0000", 2021-01-15T19:08:35.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1079999999999998e-06 2021-01-15T19:08:35.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212025+0000", 2021-01-15T19:08:35.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7986999999999999e-05 2021-01-15T19:08:35.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212083+0000", 2021-01-15T19:08:35.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1340000000000003e-05 2021-01-15T19:08:35.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.212174+0000", 2021-01-15T19:08:35.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00089311700000000002 2021-01-15T19:08:35.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:35.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.213067+0000", 2021-01-15T19:08:35.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068268399999999998 2021-01-15T19:08:35.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:35.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.219894+0000", 2021-01-15T19:08:35.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122741 2021-01-15T19:08:35.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:35.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.220017+0000", 2021-01-15T19:08:35.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010232 2021-01-15T19:08:35.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.220119+0000", 2021-01-15T19:08:35.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083155529999999998 2021-01-15T19:08:35.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.135 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.135 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.136 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:368 3.cs0 3:3ef45b66:::smithi00140323-11:14 [read 0~319488,checksum] snapc 0=[] ondisk+read+known_if_redirected e45)", 2021-01-15T19:08:35.136 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:33.191578+0000", 2021-01-15T19:08:35.136 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.653275227, 2021-01-15T19:08:35.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0078205989999999993, 2021-01-15T19:08:35.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:35.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.137 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 368 2021-01-15T19:08:35.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.138 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191578+0000", 2021-01-15T19:08:35.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191578+0000", 2021-01-15T19:08:35.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2171000000000001e-05 2021-01-15T19:08:35.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191610+0000", 2021-01-15T19:08:35.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7910000000000002e-06 2021-01-15T19:08:35.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191614+0000", 2021-01-15T19:08:35.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8029999999999999e-06 2021-01-15T19:08:35.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191621+0000", 2021-01-15T19:08:35.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8890000000000002e-06 2021-01-15T19:08:35.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191624+0000", 2021-01-15T19:08:35.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7087000000000003e-05 2021-01-15T19:08:35.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191691+0000", 2021-01-15T19:08:35.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9590000000000007e-06 2021-01-15T19:08:35.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:35.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191701+0000", 2021-01-15T19:08:35.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0035910859999999998 2021-01-15T19:08:35.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.195292+0000", 2021-01-15T19:08:35.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2689999999999998e-05 2021-01-15T19:08:35.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.195354+0000", 2021-01-15T19:08:35.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040441230000000002 2021-01-15T19:08:35.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.199399+0000", 2021-01-15T19:08:35.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0078205989999999993 2021-01-15T19:08:35.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.149 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.150 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:369 3.cs0 3:3ef45b66:::smithi00140323-11:14 [read 0~319488,checksum] snapc 0=[] ondisk+read+known_if_redirected e45)", 2021-01-15T19:08:35.150 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:33.191658+0000", 2021-01-15T19:08:35.150 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.6531950740000001, 2021-01-15T19:08:35.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007993287, 2021-01-15T19:08:35.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:35.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:35.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:35.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:35.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:35.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 369 2021-01-15T19:08:35.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:35.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:35.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191658+0000", 2021-01-15T19:08:35.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:35.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:35.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191658+0000", 2021-01-15T19:08:35.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8072999999999998e-05 2021-01-15T19:08:35.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:35.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191716+0000", 2021-01-15T19:08:35.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0099999999999997e-06 2021-01-15T19:08:35.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:35.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191720+0000", 2021-01-15T19:08:35.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2229999999999996e-06 2021-01-15T19:08:35.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:35.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191729+0000", 2021-01-15T19:08:35.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2339999999999999e-06 2021-01-15T19:08:35.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:35.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191733+0000", 2021-01-15T19:08:35.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8086000000000001e-05 2021-01-15T19:08:35.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191771+0000", 2021-01-15T19:08:35.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7870000000000007e-06 2021-01-15T19:08:35.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:35.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.191780+0000", 2021-01-15T19:08:35.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044930999999999999 2021-01-15T19:08:35.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:35.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.196273+0000", 2021-01-15T19:08:35.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000101174 2021-01-15T19:08:35.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:35.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.196374+0000", 2021-01-15T19:08:35.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032775999999999999 2021-01-15T19:08:35.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:35.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:35.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:35.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:33.199651+0000", 2021-01-15T19:08:35.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007993287 2021-01-15T19:08:35.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.163 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:35.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:35.164 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:35.166 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:35.166 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:35.164+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:35.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:35.199 INFO:tasks.rados.rados.0.smithi001.stdout:146: done (0 left) 2021-01-15T19:08:35.199 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 47 current snap is 13 2021-01-15T19:08:35.199 INFO:tasks.rados.rados.0.smithi001.stdout:147: seq_num 77 ranges {606208=491520,1097728=794624,1892352=8192} 2021-01-15T19:08:35.213 INFO:tasks.rados.rados.0.smithi001.stdout:147: writing smithi00140323-47 from 606208 to 1097728 tid 1 2021-01-15T19:08:35.219 INFO:tasks.rados.rados.0.smithi001.stdout:147: writing smithi00140323-47 from 1097728 to 1892352 tid 2 2021-01-15T19:08:35.219 INFO:tasks.rados.rados.0.smithi001.stdout:147: writing smithi00140323-47 from 1892352 to 1900544 tid 3 2021-01-15T19:08:35.219 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 48 current snap is 13 2021-01-15T19:08:35.220 INFO:tasks.rados.rados.0.smithi001.stdout:148: seq_num 78 ranges {3137536=516096,3653632=679936,4333568=122880} 2021-01-15T19:08:35.239 INFO:tasks.rados.rados.0.smithi001.stdout:148: writing smithi00140323-48 from 3137536 to 3653632 tid 1 2021-01-15T19:08:35.243 INFO:tasks.rados.rados.0.smithi001.stdout:148: writing smithi00140323-48 from 3653632 to 4333568 tid 2 2021-01-15T19:08:35.244 INFO:tasks.rados.rados.0.smithi001.stdout:148: writing smithi00140323-48 from 4333568 to 4456448 tid 3 2021-01-15T19:08:35.245 INFO:tasks.rados.rados.0.smithi001.stdout:149: snap_remove snap 11 2021-01-15T19:08:35.266 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:35.264+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:35.319 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:35.320 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:35.320 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:35.330 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:35.362 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:35.382 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:35.383 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:35.381+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:35.404 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:35.425 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:35.446 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:35.467 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:35.468 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:35.465+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:35.488 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:35.496 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:35.496 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:35.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:35.509 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.514 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.514 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.514 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.514 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:35.569 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:35.567+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:35.640 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:35.641 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:35.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:35.671 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:35.669+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:35.771 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:35.769+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:35.822 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:35.823 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:35.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:35.875 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:35.872+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:35.973 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:35.971+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:36.009 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.010 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:36.016 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:36.074 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:36.072+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:36.147 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.147 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:36.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:36.175 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:36.173+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:36.207 INFO:tasks.rados.rados.0.smithi001.stdout:149: done (2 left) 2021-01-15T19:08:36.207 INFO:tasks.rados.rados.0.smithi001.stdout:150: delete oid 18 current snap is 13 2021-01-15T19:08:36.207 INFO:tasks.rados.rados.0.smithi001.stdout:147: finishing write tid 1 to smithi00140323-47 2021-01-15T19:08:36.207 INFO:tasks.rados.rados.0.smithi001.stdout:147: finishing write tid 2 to smithi00140323-47 2021-01-15T19:08:36.208 INFO:tasks.rados.rados.0.smithi001.stdout:147: finishing write tid 3 to smithi00140323-47 2021-01-15T19:08:36.208 INFO:tasks.rados.rados.0.smithi001.stdout:147: finishing write tid 4 to smithi00140323-47 2021-01-15T19:08:36.208 INFO:tasks.rados.rados.0.smithi001.stdout:147: finishing write tid 5 to smithi00140323-47 2021-01-15T19:08:36.208 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 39 (ObjNum 77 snap 13 seq_num 77) dirty exists 2021-01-15T19:08:36.208 INFO:tasks.rados.rados.0.smithi001.stdout:147: left oid 47 (ObjNum 77 snap 13 seq_num 77) 2021-01-15T19:08:36.209 INFO:tasks.rados.rados.0.smithi001.stdout:148: finishing write tid 1 to smithi00140323-48 2021-01-15T19:08:36.209 INFO:tasks.rados.rados.0.smithi001.stdout:148: finishing write tid 2 to smithi00140323-48 2021-01-15T19:08:36.209 INFO:tasks.rados.rados.0.smithi001.stdout:148: finishing write tid 3 to smithi00140323-48 2021-01-15T19:08:36.209 INFO:tasks.rados.rados.0.smithi001.stdout:148: finishing write tid 4 to smithi00140323-48 2021-01-15T19:08:36.209 INFO:tasks.rados.rados.0.smithi001.stdout:148: finishing write tid 5 to smithi00140323-48 2021-01-15T19:08:36.210 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 26 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:08:36.210 INFO:tasks.rados.rados.0.smithi001.stdout:148: left oid 48 (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:08:36.216 INFO:tasks.rados.rados.0.smithi001.stdout:147: done (2 left) 2021-01-15T19:08:36.217 INFO:tasks.rados.rados.0.smithi001.stdout:148: done (1 left) 2021-01-15T19:08:36.217 INFO:tasks.rados.rados.0.smithi001.stdout:150: done (0 left) 2021-01-15T19:08:36.217 INFO:tasks.rados.rados.0.smithi001.stdout:151: rollback oid 17 current snap is 13 2021-01-15T19:08:36.217 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 17 to 10 2021-01-15T19:08:36.218 INFO:tasks.rados.rados.0.smithi001.stdout:152: snap_create 2021-01-15T19:08:36.224 INFO:tasks.rados.rados.0.smithi001.stdout:151: finishing rollback tid 0 to smithi00140323-17 2021-01-15T19:08:36.225 INFO:tasks.rados.rados.0.smithi001.stdout:151: finishing rollback tid 1 to smithi00140323-17 2021-01-15T19:08:36.226 INFO:tasks.rados.rados.0.smithi001.stdout:151: finishing rollback tid 2 to smithi00140323-17 2021-01-15T19:08:36.226 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 17 v 16 (ObjNum 16 snap 0 seq_num 16) dirty exists 2021-01-15T19:08:36.276 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:36.274+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:36.298 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.299 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:36.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:36.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:08:36.394 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:36.393+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:36.481 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:36.479+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:36.485 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.486 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:36.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:36.581 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:36.580+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:36.670 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.670 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:36.680 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:36.720 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:36.719+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:36.782 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:36.780+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:36.819 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.819 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:36.828 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:36.912 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:36.910+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:36.983 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:36.981+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:36.995 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:36.995 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:37.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:37.084 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:37.082+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:37.170 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:37.170 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:37.180 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:37.208 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:37.208 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:37.209 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2021-01-15T19:08:37.209 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:37.212 INFO:tasks.rados.rados.0.smithi001.stdout:151: done (1 left) 2021-01-15T19:08:37.212 INFO:tasks.rados.rados.0.smithi001.stdout:152: done (0 left) 2021-01-15T19:08:37.212 INFO:tasks.rados.rados.0.smithi001.stdout:153: copy_from oid 27 from oid 38 current snap is 14 2021-01-15T19:08:37.212 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 28 current snap is 14 2021-01-15T19:08:37.213 INFO:tasks.rados.rados.0.smithi001.stdout:154: seq_num 80 ranges {458752=548864,1007616=450560,1458176=352256} 2021-01-15T19:08:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:08:37.218 INFO:tasks.rados.rados.0.smithi001.stdout:154: writing smithi00140323-28 from 458752 to 1007616 tid 1 2021-01-15T19:08:37.221 INFO:tasks.rados.rados.0.smithi001.stdout:154: writing smithi00140323-28 from 1007616 to 1458176 tid 2 2021-01-15T19:08:37.225 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:37.223+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:37.226 INFO:tasks.rados.rados.0.smithi001.stdout:154: writing smithi00140323-28 from 1458176 to 1810432 tid 3 2021-01-15T19:08:37.227 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 25 current snap is 14 2021-01-15T19:08:37.227 INFO:tasks.rados.rados.0.smithi001.stdout:155: seq_num 81 ranges {409600=565248,974848=737280,1712128=65536} 2021-01-15T19:08:37.232 INFO:tasks.rados.rados.0.smithi001.stdout:155: writing smithi00140323-25 from 409600 to 974848 tid 1 2021-01-15T19:08:37.236 INFO:tasks.rados.rados.0.smithi001.stdout:155: writing smithi00140323-25 from 974848 to 1712128 tid 2 2021-01-15T19:08:37.237 INFO:tasks.rados.rados.0.smithi001.stdout:155: writing smithi00140323-25 from 1712128 to 1777664 tid 3 2021-01-15T19:08:37.237 INFO:tasks.rados.rados.0.smithi001.stdout:153: finishing copy_from to smithi00140323-27 2021-01-15T19:08:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 27 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2021-01-15T19:08:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 3 current snap is 14 2021-01-15T19:08:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:156: seq_num 82 ranges {40960=581632,622592=622592,1245184=180224} 2021-01-15T19:08:37.241 INFO:tasks.rados.rados.0.smithi001.stdout:156: writing smithi00140323-3 from 40960 to 622592 tid 1 2021-01-15T19:08:37.245 INFO:tasks.rados.rados.0.smithi001.stdout:156: writing smithi00140323-3 from 622592 to 1245184 tid 2 2021-01-15T19:08:37.246 INFO:tasks.rados.rados.0.smithi001.stdout:156: writing smithi00140323-3 from 1245184 to 1425408 tid 3 2021-01-15T19:08:37.246 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 30 current snap is 14 2021-01-15T19:08:37.247 INFO:tasks.rados.rados.0.smithi001.stdout:153: finishing copy_from racing read to smithi00140323-27 2021-01-15T19:08:37.247 INFO:tasks.rados.rados.0.smithi001.stdout:154: finishing write tid 1 to smithi00140323-28 2021-01-15T19:08:37.247 INFO:tasks.rados.rados.0.smithi001.stdout:154: finishing write tid 2 to smithi00140323-28 2021-01-15T19:08:37.247 INFO:tasks.rados.rados.0.smithi001.stdout:154: finishing write tid 3 to smithi00140323-28 2021-01-15T19:08:37.247 INFO:tasks.rados.rados.0.smithi001.stdout:154: finishing write tid 4 to smithi00140323-28 2021-01-15T19:08:37.248 INFO:tasks.rados.rados.0.smithi001.stdout:155: finishing write tid 1 to smithi00140323-25 2021-01-15T19:08:37.248 INFO:tasks.rados.rados.0.smithi001.stdout:154: finishing write tid 5 to smithi00140323-28 2021-01-15T19:08:37.248 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 23 (ObjNum 80 snap 14 seq_num 80) dirty exists 2021-01-15T19:08:37.248 INFO:tasks.rados.rados.0.smithi001.stdout:154: left oid 28 (ObjNum 80 snap 14 seq_num 80) 2021-01-15T19:08:37.248 INFO:tasks.rados.rados.0.smithi001.stdout:157: seq_num 83 ranges {491520=598016,1089536=499712,1589248=303104} 2021-01-15T19:08:37.252 INFO:tasks.rados.rados.0.smithi001.stdout:157: writing smithi00140323-30 from 491520 to 1089536 tid 1 2021-01-15T19:08:37.255 INFO:tasks.rados.rados.0.smithi001.stdout:157: writing smithi00140323-30 from 1089536 to 1589248 tid 2 2021-01-15T19:08:37.257 INFO:tasks.rados.rados.0.smithi001.stdout:157: writing smithi00140323-30 from 1589248 to 1892352 tid 3 2021-01-15T19:08:37.257 INFO:tasks.rados.rados.0.smithi001.stdout:153: done (4 left) 2021-01-15T19:08:37.257 INFO:tasks.rados.rados.0.smithi001.stdout:154: done (3 left) 2021-01-15T19:08:37.258 INFO:tasks.rados.rados.0.smithi001.stdout:158: snap_remove snap 9 2021-01-15T19:08:37.287 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:37.285+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:37.327 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:37.327 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:37.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:37.420 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:37.418+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:37.489 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:37.488+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:37.502 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:37.502 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:37.512 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:37.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:37.540 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:37.551 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2021-01-15T19:08:37.513200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48742400,"num_objects":68,"num_object_clones":17,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":1,"num_read":216,"num_read_kb":30521,"num_write":289,"num_write_kb":51401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":120892,"kb_used_data":71932,"kb_used_omap":0,"kb_used_meta":48896,"kb_avail":754853828,"statfs":{"total":773094113280,"available":772970319872,"internally_reserved":0,"allocated":73658368,"data_stored":89923606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50069504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8830976,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":15,"num_read_kb":1571,"num_write":26,"num_write_kb":7504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000726"},"pg_stats":[{"pgid":"2.11","version":"0'0","reported_seq":"1","reported_epoch":"48","state":"peering","last_fresh":"2021-01-15T19:08:36.216096+0000","last_change":"2021-01-15T19:08:36.216096+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:36.216096+0000","last_undegraded":"2021-01-15T19:08:36.216096+0000","last_fullsized":"2021-01-15T19:08:36.216096+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:34.188123+0000","last_change":"2021-01-15T19:08:34.188123+0000","last_active":"2021-01-15T19:08:34.188123+0000","last_peered":"2021-01-15T19:08:34.188123+0000","last_clean":"2021-01-15T19:08:34.188123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:34.188123+0000","last_undegraded":"2021-01-15T19:08:34.188123+0000","last_fullsized":"2021-01-15T19:08:34.188123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_clean_scrub_stamp":"2021-01-15T19:08:34.188123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":"45","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.205374+0000","last_change":"2021-01-15T19:08:35.205374+0000","last_active":"2021-01-15T19:08:35.205374+0000","last_peered":"2021-01-15T19:08:35.205374+0000","last_clean":"2021-01-15T19:08:35.205374+0000","last_became_active":"2021-01-15T19:08:35.204902+0000","last_became_peered":"2021-01-15T19:08:35.204902+0000","last_unstale":"2021-01-15T19:08:35.205374+0000","last_undegraded":"2021-01-15T19:08:35.205374+0000","last_fullsized":"2021-01-15T19:08:35.205374+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"40'30","reported_seq":"113","reported_epoch":"48","state":"active+clean","last_fresh":"2021-01-15T19:08:36.214574+0000","last_change":"2021-01-15T19:08:36.214851+0000","last_active":"2021-01-15T19:08:36.214574+0000","last_peered":"2021-01-15T19:08:36.214574+0000","last_clean":"2021-01-15T19:08:36.214574+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:36.214574+0000","last_undegraded":"2021-01-15T19:08:36.214574+0000","last_fullsized":"2021-01-15T19:08:36.214574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3776512,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":2623,"num_write":33,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"15","length":"2"}]},{"pgid":"2.f","version":"0'0","reported_seq":"37","reported_epoch":"48","state":"peering","last_fresh":"2021-01-15T19:08:36.213486+0000","last_change":"2021-01-15T19:08:36.213737+0000","last_active":"2021-01-15T19:08:34.196711+0000","last_peered":"2021-01-15T19:08:33.956484+0000","last_clean":"2021-01-15T19:08:33.956484+0000","last_became_active":"2021-01-15T19:07:56.779106+0000","last_became_peered":"2021-01-15T19:07:56.779106+0000","last_unstale":"2021-01-15T19:08:36.213486+0000","last_undegraded":"2021-01-15T19:08:36.213486+0000","last_fullsized":"2021-01-15T19:08:36.213486+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"39'8","reported_seq":"86","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.198530+0000","last_change":"2021-01-15T19:08:35.199346+0000","last_active":"2021-01-15T19:08:35.198530+0000","last_peered":"2021-01-15T19:08:35.198530+0000","last_clean":"2021-01-15T19:08:35.198530+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:35.198530+0000","last_undegraded":"2021-01-15T19:08:35.198530+0000","last_fullsized":"2021-01-15T19:08:35.198530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":753664,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":555,"num_write":5,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"43","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.213946+0000","last_change":"2021-01-15T19:08:35.213946+0000","last_active":"2021-01-15T19:08:35.213946+0000","last_peered":"2021-01-15T19:08:35.213946+0000","last_clean":"2021-01-15T19:08:35.213946+0000","last_became_active":"2021-01-15T19:08:35.213564+0000","last_became_peered":"2021-01-15T19:08:35.213564+0000","last_unstale":"2021-01-15T19:08:35.213946+0000","last_undegraded":"2021-01-15T19:08:35.213946+0000","last_fullsized":"2021-01-15T19:08:35.213946+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"39'12","reported_seq":"97","reported_epoch":"48","state":"active+clean","last_fresh":"2021-01-15T19:08:36.210436+0000","last_change":"2021-01-15T19:08:36.210670+0000","last_active":"2021-01-15T19:08:36.210436+0000","last_peered":"2021-01-15T19:08:36.210436+0000","last_clean":"2021-01-15T19:08:36.210436+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:36.210436+0000","last_undegraded":"2021-01-15T19:08:36.210436+0000","last_fullsized":"2021-01-15T19:08:36.210436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":417792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":293,"num_write":26,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"15","length":"2"}]},{"pgid":"2.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:34.188123+0000","last_change":"2021-01-15T19:08:34.188123+0000","last_active":"2021-01-15T19:08:34.188123+0000","last_peered":"2021-01-15T19:08:34.188123+0000","last_clean":"2021-01-15T19:08:34.188123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:34.188123+0000","last_undegraded":"2021-01-15T19:08:34.188123+0000","last_fullsized":"2021-01-15T19:08:34.188123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_clean_scrub_stamp":"2021-01-15T19:08:34.188123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"40'38","reported_seq":"124","reported_epoch":"48","state":"active+clean","last_fresh":"2021-01-15T19:08:36.213524+0000","last_change":"2021-01-15T19:08:36.213917+0000","last_active":"2021-01-15T19:08:36.213524+0000","last_peered":"2021-01-15T19:08:36.213524+0000","last_clean":"2021-01-15T19:08:36.213524+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:36.213524+0000","last_undegraded":"2021-01-15T19:08:36.213524+0000","last_fullsized":"2021-01-15T19:08:36.213524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4849664,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":2607,"num_write":24,"num_write_kb":6256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"15","length":"2"}]},{"pgid":"2.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:34.188123+0000","last_change":"2021-01-15T19:08:34.188123+0000","last_active":"2021-01-15T19:08:34.188123+0000","last_peered":"2021-01-15T19:08:34.188123+0000","last_clean":"2021-01-15T19:08:34.188123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:34.188123+0000","last_undegraded":"2021-01-15T19:08:34.188123+0000","last_fullsized":"2021-01-15T19:08:34.188123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_clean_scrub_stamp":"2021-01-15T19:08:34.188123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"25'4","reported_seq":"90","reported_epoch":"48","state":"active+clean","last_fresh":"2021-01-15T19:08:36.213626+0000","last_change":"2021-01-15T19:08:36.213979+0000","last_active":"2021-01-15T19:08:36.213626+0000","last_peered":"2021-01-15T19:08:36.213626+0000","last_clean":"2021-01-15T19:08:36.213626+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:36.213626+0000","last_undegraded":"2021-01-15T19:08:36.213626+0000","last_fullsized":"2021-01-15T19:08:36.213626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1220608,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":4276,"num_write":4,"num_write_kb":1192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"15","length":"2"}]},{"pgid":"2.b","version":"0'0","reported_seq":"43","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.214660+0000","last_change":"2021-01-15T19:08:35.210579+0000","last_active":"2021-01-15T19:08:35.214660+0000","last_peered":"2021-01-15T19:08:35.214660+0000","last_clean":"2021-01-15T19:08:35.214660+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:08:35.214660+0000","last_undegraded":"2021-01-15T19:08:35.214660+0000","last_fullsized":"2021-01-15T19:08:35.214660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"48'28","reported_seq":"108","reported_epoch":"48","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:08:36.210024+0000","last_change":"2021-01-15T19:08:36.210024+0000","last_active":"2021-01-15T19:08:36.210024+0000","last_peered":"2021-01-15T19:08:36.210024+0000","last_clean":"2021-01-15T19:08:36.210024+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:36.210024+0000","last_undegraded":"2021-01-15T19:08:36.210024+0000","last_fullsized":"2021-01-15T19:08:36.210024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":6750208,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":1688,"num_write":20,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"1","reported_epoch":"48","state":"peering","last_fresh":"2021-01-15T19:08:36.213782+0000","last_change":"2021-01-15T19:08:36.213782+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:36.213782+0000","last_undegraded":"2021-01-15T19:08:36.213782+0000","last_fullsized":"2021-01-15T19:08:36.213782+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"48'19","reported_seq":"105","reported_epoch":"48","state":"active+clean","last_fresh":"2021-01-15T19:08:36.210359+0000","last_change":"2021-01-15T19:08:36.210359+0000","last_active":"2021-01-15T19:08:36.210359+0000","last_peered":"2021-01-15T19:08:36.210359+0000","last_clean":"2021-01-15T19:08:36.210359+0000","last_became_active":"2021-01-15T19:08:10.230943+0000","last_became_peered":"2021-01-15T19:08:10.230943+0000","last_unstale":"2021-01-15T19:08:36.210359+0000","last_undegraded":"2021-01-15T19:08:36.210359+0000","last_fullsized":"2021-01-15T19:08:36.210359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3129344,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":19,"num_read_kb":2936,"num_write":15,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"15","length":"2"}]},{"pgid":"2.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:34.188123+0000","last_change":"2021-01-15T19:08:34.188123+0000","last_active":"2021-01-15T19:08:34.188123+0000","last_peered":"2021-01-15T19:08:34.188123+0000","last_clean":"2021-01-15T19:08:34.188123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:34.188123+0000","last_undegraded":"2021-01-15T19:08:34.188123+0000","last_fullsized":"2021-01-15T19:08:34.188123+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:34.188123+0000","last_clean_scrub_stamp":"2021-01-15T19:08:34.188123+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"39'18","reported_seq":"98","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.199729+0000","last_change":"2021-01-15T19:08:35.200100+0000","last_active":"2021-01-15T19:08:35.199729+0000","last_peered":"2021-01-15T19:08:35.199729+0000","last_clean":"2021-01-15T19:08:35.199729+0000","last_became_active":"2021-01-15T19:08:10.231245+0000","last_became_peered":"2021-01-15T19:08:10.231245+0000","last_unstale":"2021-01-15T19:08:35.199729+0000","last_undegraded":"2021-01-15T19:08:35.199729+0000","last_fullsized":"2021-01-15T19:08:35.199729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2547712,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":607,"num_write":12,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"44","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.205924+0000","last_change":"2021-01-15T19:08:35.205924+0000","last_active":"2021-01-15T19:08:35.205924+0000","last_peered":"2021-01-15T19:08:35.205924+0000","last_clean":"2021-01-15T19:08:35.205924+0000","last_became_active":"2021-01-15T19:08:35.204979+0000","last_became_peered":"2021-01-15T19:08:35.204979+0000","last_unstale":"2021-01-15T19:08:35.205924+0000","last_undegraded":"2021-01-15T19:08:35.205924+0000","last_fullsized":"2021-01-15T19:08:35.205924+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"45'12","reported_seq":"84","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.198440+0000","last_change":"2021-01-15T19:08:35.198823+0000","last_active":"2021-01-15T19:08:35.198440+0000","last_peered":"2021-01-15T19:08:35.198440+0000","last_clean":"2021-01-15T19:08:35.198440+0000","last_became_active":"2021-01-15T19:08:10.230610+0000","last_became_peered":"2021-01-15T19:08:10.230610+0000","last_unstale":"2021-01-15T19:08:35.198440+0000","last_undegraded":"2021-01-15T19:08:35.198440+0000","last_fullsized":"2021-01-15T19:08:35.198440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1548288,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":28,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"44","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.215139+0000","last_change":"2021-01-15T19:08:35.214301+0000","last_active":"2021-01-15T19:08:35.215139+0000","last_peered":"2021-01-15T19:08:35.215139+0000","last_clean":"2021-01-15T19:08:35.215139+0000","last_became_active":"2021-01-15T19:08:35.213894+0000","last_became_peered":"2021-01-15T19:08:35.213894+0000","last_unstale":"2021-01-15T19:08:35.215139+0000","last_undegraded":"2021-01-15T19:08:35.215139+0000","last_fullsized":"2021-01-15T19:08:35.215139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"75","reported_epoch":"48","state":"active+clean","last_fresh":"2021-01-15T19:08:36.213429+0000","last_change":"2021-01-15T19:08:36.213729+0000","last_active":"2021-01-15T19:08:36.213429+0000","last_peered":"2021-01-15T19:08:36.213429+0000","last_clean":"2021-01-15T19:08:36.213429+0000","last_became_active":"2021-01-15T19:08:10.237812+0000","last_became_peered":"2021-01-15T19:08:10.237812+0000","last_unstale":"2021-01-15T19:08:36.213429+0000","last_undegraded":"2021-01-15T19:08:36.213429+0000","last_fullsized":"2021-01-15T19:08:36.213429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"15","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":"55","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.212156+0000","last_change":"2021-01-15T19:08:35.204482+0000","last_active":"2021-01-15T19:08:35.212156+0000","last_peered":"2021-01-15T19:08:35.212156+0000","last_clean":"2021-01-15T19:08:35.212156+0000","last_became_active":"2021-01-15T19:08:35.204089+0000","last_became_peered":"2021-01-15T19:08:35.204089+0000","last_unstale":"2021-01-15T19:08:35.212156+0000","last_undegraded":"2021-01-15T19:08:35.212156+0000","last_fullsized":"2021-01-15T19:08:35.212156+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"47'39","reported_seq":"125","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.226657+0000","last_change":"2021-01-15T19:08:35.199466+0000","last_active":"2021-01-15T19:08:35.226657+0000","last_peered":"2021-01-15T19:08:35.226657+0000","last_clean":"2021-01-15T19:08:35.226657+0000","last_became_active":"2021-01-15T19:08:10.231211+0000","last_became_peered":"2021-01-15T19:08:10.231211+0000","last_unstale":"2021-01-15T19:08:35.226657+0000","last_undegraded":"2021-01-15T19:08:35.226657+0000","last_fullsized":"2021-01-15T19:08:35.226657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5603328,"num_objects":8,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":28,"num_read_kb":1931,"num_write":26,"num_write_kb":6480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":"44","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.214817+0000","last_change":"2021-01-15T19:08:35.211897+0000","last_active":"2021-01-15T19:08:35.214817+0000","last_peered":"2021-01-15T19:08:35.214817+0000","last_clean":"2021-01-15T19:08:35.214817+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:08:35.214817+0000","last_undegraded":"2021-01-15T19:08:35.214817+0000","last_fullsized":"2021-01-15T19:08:35.214817+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":"43","reported_epoch":"46","state":"active+clean","last_fresh":"2021-01-15T19:08:34.848287+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:34.848287+0000","last_peered":"2021-01-15T19:08:34.848287+0000","last_clean":"2021-01-15T19:08:34.848287+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:34.848287+0000","last_undegraded":"2021-01-15T19:08:34.848287+0000","last_fullsized":"2021-01-15T19:08:34.848287+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"28'12","reported_seq":"91","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.199851+0000","last_change":"2021-01-15T19:08:35.200181+0000","last_active":"2021-01-15T19:08:35.199851+0000","last_peered":"2021-01-15T19:08:35.199851+0000","last_clean":"2021-01-15T19:08:35.199851+0000","last_became_active":"2021-01-15T19:08:10.234192+0000","last_became_peered":"2021-01-15T19:08:10.234192+0000","last_unstale":"2021-01-15T19:08:35.199851+0000","last_undegraded":"2021-01-15T19:08:35.199851+0000","last_fullsized":"2021-01-15T19:08:35.199851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1974272,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3221,"num_write":9,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"44","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.214812+0000","last_change":"2021-01-15T19:08:35.211611+0000","last_active":"2021-01-15T19:08:35.214812+0000","last_peered":"2021-01-15T19:08:35.214812+0000","last_clean":"2021-01-15T19:08:35.214812+0000","last_became_active":"2021-01-15T19:08:35.210980+0000","last_became_peered":"2021-01-15T19:08:35.210980+0000","last_unstale":"2021-01-15T19:08:35.214812+0000","last_undegraded":"2021-01-15T19:08:35.214812+0000","last_fullsized":"2021-01-15T19:08:35.214812+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"43'35","reported_seq":"109","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.202405+0000","last_change":"2021-01-15T19:08:35.202764+0000","last_active":"2021-01-15T19:08:35.202405+0000","last_peered":"2021-01-15T19:08:35.202405+0000","last_clean":"2021-01-15T19:08:35.202405+0000","last_became_active":"2021-01-15T19:08:10.230297+0000","last_became_peered":"2021-01-15T19:08:10.230297+0000","last_unstale":"2021-01-15T19:08:35.202405+0000","last_undegraded":"2021-01-15T19:08:35.202405+0000","last_fullsized":"2021-01-15T19:08:35.202405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5996544,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":1316,"num_write":38,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"44","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.218146+0000","last_change":"2021-01-15T19:08:35.215835+0000","last_active":"2021-01-15T19:08:35.218146+0000","last_peered":"2021-01-15T19:08:35.218146+0000","last_clean":"2021-01-15T19:08:35.218146+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:08:35.218146+0000","last_undegraded":"2021-01-15T19:08:35.218146+0000","last_fullsized":"2021-01-15T19:08:35.218146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"39'29","reported_seq":"106","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.203278+0000","last_change":"2021-01-15T19:08:35.203618+0000","last_active":"2021-01-15T19:08:35.203278+0000","last_peered":"2021-01-15T19:08:35.203278+0000","last_clean":"2021-01-15T19:08:35.203278+0000","last_became_active":"2021-01-15T19:08:10.228436+0000","last_became_peered":"2021-01-15T19:08:10.228436+0000","last_unstale":"2021-01-15T19:08:35.203278+0000","last_undegraded":"2021-01-15T19:08:35.203278+0000","last_fullsized":"2021-01-15T19:08:35.203278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4243456,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":8047,"num_write":15,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"3.7","version":"45'26","reported_seq":"99","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.200742+0000","last_change":"2021-01-15T19:08:35.201086+0000","last_active":"2021-01-15T19:08:35.200742+0000","last_peered":"2021-01-15T19:08:35.200742+0000","last_clean":"2021-01-15T19:08:35.200742+0000","last_became_active":"2021-01-15T19:08:10.703898+0000","last_became_peered":"2021-01-15T19:08:10.703898+0000","last_unstale":"2021-01-15T19:08:35.200742+0000","last_undegraded":"2021-01-15T19:08:35.200742+0000","last_fullsized":"2021-01-15T19:08:35.200742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3039232,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":15,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":"46","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.206509+0000","last_change":"2021-01-15T19:08:35.206509+0000","last_active":"2021-01-15T19:08:35.206509+0000","last_peered":"2021-01-15T19:08:35.206509+0000","last_clean":"2021-01-15T19:08:35.206509+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:08:35.206509+0000","last_undegraded":"2021-01-15T19:08:35.206509+0000","last_fullsized":"2021-01-15T19:08:35.206509+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"44","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.214610+0000","last_change":"2021-01-15T19:08:35.209452+0000","last_active":"2021-01-15T19:08:35.214610+0000","last_peered":"2021-01-15T19:08:35.214610+0000","last_clean":"2021-01-15T19:08:35.214610+0000","last_became_active":"2021-01-15T19:08:35.208618+0000","last_became_peered":"2021-01-15T19:08:35.208618+0000","last_unstale":"2021-01-15T19:08:35.214610+0000","last_undegraded":"2021-01-15T19:08:35.214610+0000","last_fullsized":"2021-01-15T19:08:35.214610+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'30","reported_seq":"107","reported_epoch":"47","state":"active+clean","last_fresh":"2021-01-15T19:08:35.201911+0000","last_change":"2021-01-15T19:08:35.202251+0000","last_active":"2021-01-15T19:08:35.201911+0000","last_peered":"2021-01-15T19:08:35.201911+0000","last_clean":"2021-01-15T19:08:35.201911+0000","last_became_active":"2021-01-15T19:08:10.703953+0000","last_became_peered":"2021-01-15T19:08:10.703953+0000","last_unstale":"2021-01-15T19:08:35.201911+0000","last_undegraded":"2021-01-15T19:08:35.201911+0000","last_fullsized":"2021-01-15T19:08:35.201911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2891776,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":414,"num_write":17,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"7","length":"1"},{"start":"11","length":"1"},{"start":"16","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":48742400,"num_objects":66,"num_object_clones":17,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":216,"num_read_kb":30521,"num_write":287,"num_write_kb":51392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70762496,"data_stored":90374144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":17880,"kb_used_data":12176,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":94353960,"statfs":{"total":96636764160,"available":96618455040,"internally_reserved":0,"allocated":12468224,"data_stored":16236976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14196,"kb_used_data":7788,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":94357644,"statfs":{"total":96636764160,"available":96622227456,"internally_reserved":0,"allocated":7974912,"data_stored":9705372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14184,"kb_used_data":8096,"kb_used_omap":0,"kb_used_meta":6080,"kb_avail":94357656,"statfs":{"total":96636764160,"available":96622239744,"internally_reserved":0,"allocated":8290304,"data_stored":10595478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6225920},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6739999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12624,"kb_used_data":6664,"kb_used_omap":0,"kb_used_meta":5952,"kb_avail":94359216,"statfs":{"total":96636764160,"available":96623837184,"internally_reserved":0,"allocated":6823936,"data_stored":7971248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6094848},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.732}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16316,"kb_used_data":9844,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":94355524,"statfs":{"total":96636764160,"available":96620056576,"internally_reserved":0,"allocated":10080256,"data_stored":11544476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.472}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13496,"kb_used_data":7920,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":94358344,"statfs":{"total":96636764160,"available":96622944256,"internally_reserved":0,"allocated":8110080,"data_stored":10830256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.379}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21820,"kb_used_data":14388,"kb_used_omap":0,"kb_used_meta":7424,"kb_avail":94350020,"statfs":{"total":96636764160,"available":96614420480,"internally_reserved":0,"allocated":14733312,"data_stored":17528732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7602176},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10376,"kb_used_data":5056,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":94361464,"statfs":{"total":96636764160,"available":96626139136,"internally_reserved":0,"allocated":5177344,"data_stored":5511068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]}]}]}} 2021-01-15T19:08:37.554 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2021-01-15T19:08:37.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2021-01-15T19:08:37.597 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:37.595+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:37.679 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:37.680 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:37.689 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:37.716 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:37.730 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:37.729+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:37.735 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:37.756 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:37.778 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:37.792 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:37.791+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:37.797 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:37.818 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:37.819 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.819 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:37.856 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:37.857 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:37.865 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:37.893 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:37.891+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:37.990 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:37.991 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:37.993 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:37.992+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:38.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:38.095 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:38.093+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:38.127 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=71, reqid=client.4418.0:183, at_version=27'8, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.873427402000001, 2021-01-15T19:08:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003, 2021-01-15T19:08:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190392 2021-01-15T19:08:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251887+0000", 2021-01-15T19:08:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6526999999999999e-05 2021-01-15T19:08:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251914+0000", 2021-01-15T19:08:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8068999999999999e-05 2021-01-15T19:08:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251932+0000", 2021-01-15T19:08:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7269999999999996e-06 2021-01-15T19:08:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251940+0000", 2021-01-15T19:08:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012910800000000001 2021-01-15T19:08:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252070+0000", 2021-01-15T19:08:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2417e-05 2021-01-15T19:08:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252082+0000", 2021-01-15T19:08:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050434080000000001 2021-01-15T19:08:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257125+0000", 2021-01-15T19:08:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6032999999999998e-05 2021-01-15T19:08:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257171+0000", 2021-01-15T19:08:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003 2021-01-15T19:08:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.869576042999999, 2021-01-15T19:08:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=79, reqid=client.4418.0:185, at_version=27'10, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.868381083999999, 2021-01-15T19:08:38.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999, 2021-01-15T19:08:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5377000000000001e-05 2021-01-15T19:08:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256768+0000", 2021-01-15T19:08:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3387000000000001e-05 2021-01-15T19:08:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256782+0000", 2021-01-15T19:08:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6764999999999998e-05 2021-01-15T19:08:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256819+0000", 2021-01-15T19:08:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256826+0000", 2021-01-15T19:08:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9475999999999998e-05 2021-01-15T19:08:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256886+0000", 2021-01-15T19:08:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1929e-05 2021-01-15T19:08:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256898+0000", 2021-01-15T19:08:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055282589999999998 2021-01-15T19:08:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262426+0000", 2021-01-15T19:08:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2537999999999997e-05 2021-01-15T19:08:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262479+0000", 2021-01-15T19:08:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999 2021-01-15T19:08:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:38.173 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.866849403, 2021-01-15T19:08:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.175 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:38.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:38.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:38.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:38.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:38.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:38.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:38.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:38.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:38.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:38.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:38.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.865925193999999, 2021-01-15T19:08:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:38.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.190 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.203 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:38.204 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.865701230999999, 2021-01-15T19:08:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:38.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.206 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:38.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:38.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:38.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:38.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:38.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:38.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:38.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:38.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:38.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:38.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=81, reqid=client.4418.0:185, at_version=0'0, trim_to=0'0, roll_forward_to=27'10))", 2021-01-15T19:08:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.862125123999999, 2021-01-15T19:08:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002, 2021-01-15T19:08:38.219 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262999+0000", 2021-01-15T19:08:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4539999999999999e-05 2021-01-15T19:08:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263024+0000", 2021-01-15T19:08:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9600000000000006e-06 2021-01-15T19:08:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263032+0000", 2021-01-15T19:08:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2021-01-15T19:08:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263041+0000", 2021-01-15T19:08:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3239999999999996e-06 2021-01-15T19:08:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263046+0000", 2021-01-15T19:08:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4546999999999999e-05 2021-01-15T19:08:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263090+0000", 2021-01-15T19:08:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0596e-05 2021-01-15T19:08:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263101+0000", 2021-01-15T19:08:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052094810000000002 2021-01-15T19:08:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268310+0000", 2021-01-15T19:08:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4725000000000003e-05 2021-01-15T19:08:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268355+0000", 2021-01-15T19:08:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053559430000000002 2021-01-15T19:08:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 27/21 ECSubWriteReply(tid=29, last_complete=27'10, committed=1, applied=1))", 2021-01-15T19:08:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.861838645999999, 2021-01-15T19:08:38.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999, 2021-01-15T19:08:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263285+0000", 2021-01-15T19:08:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3807000000000001e-05 2021-01-15T19:08:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263309+0000", 2021-01-15T19:08:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263317+0000", 2021-01-15T19:08:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5400000000000001e-06 2021-01-15T19:08:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263327+0000", 2021-01-15T19:08:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9009999999999997e-06 2021-01-15T19:08:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263334+0000", 2021-01-15T19:08:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052278899999999998 2021-01-15T19:08:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268562+0000", 2021-01-15T19:08:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4100000000000001e-05 2021-01-15T19:08:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268576+0000", 2021-01-15T19:08:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052904179999999999 2021-01-15T19:08:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.240 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:38.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.841342198, 2021-01-15T19:08:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.243 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:38.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:38.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:38.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:38.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:38.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:38.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:38.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:38.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:38.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:38.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:38.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.256 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.839162994, 2021-01-15T19:08:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.257 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:38.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:38.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:38.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.83840863, 2021-01-15T19:08:38.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.273 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:38.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:38.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:38.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:38.288 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.838288464, 2021-01-15T19:08:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:38.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:38.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:38.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:38.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:38.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:38.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:38.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:38.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:38.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:38.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:38.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:38.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:38.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:38.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:38.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:38.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:38.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:38.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:38.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:38.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:38.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:38.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:38.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:38.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:318 3.1s0 3:8b30e322:::smithi00140323-24:head [copy-from ver 8] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:38.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:38.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9290878669999998, 2021-01-15T19:08:38.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729, 2021-01-15T19:08:38.306 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.306 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.307 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 318 2021-01-15T19:08:38.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.307 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:38.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:38.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2021-01-15T19:08:38.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196102+0000", 2021-01-15T19:08:38.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6286e-05 2021-01-15T19:08:38.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196119+0000", 2021-01-15T19:08:38.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0645e-05 2021-01-15T19:08:38.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196139+0000", 2021-01-15T19:08:38.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576000000000001e-05 2021-01-15T19:08:38.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196151+0000", 2021-01-15T19:08:38.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017478200000000001 2021-01-15T19:08:38.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196326+0000", 2021-01-15T19:08:38.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019917200000000001 2021-01-15T19:08:38.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196525+0000", 2021-01-15T19:08:38.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099001019999999992 2021-01-15T19:08:38.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.206425+0000", 2021-01-15T19:08:38.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043897240000000002 2021-01-15T19:08:38.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210815+0000", 2021-01-15T19:08:38.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023466619999999998 2021-01-15T19:08:38.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213161+0000", 2021-01-15T19:08:38.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108811 2021-01-15T19:08:38.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213270+0000", 2021-01-15T19:08:38.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729 2021-01-15T19:08:38.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:319 3.1s0 3:8b30e322:::smithi00140323-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:38.322 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:38.323 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9289249490000007, 2021-01-15T19:08:38.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001, 2021-01-15T19:08:38.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.323 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:38.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.324 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.324 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 319 2021-01-15T19:08:38.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:38.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:38.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6667e-05 2021-01-15T19:08:38.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196236+0000", 2021-01-15T19:08:38.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:38.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196243+0000", 2021-01-15T19:08:38.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4992e-05 2021-01-15T19:08:38.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196258+0000", 2021-01-15T19:08:38.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6110000000000001e-06 2021-01-15T19:08:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196264+0000", 2021-01-15T19:08:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046740399999999999 2021-01-15T19:08:38.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196731+0000", 2021-01-15T19:08:38.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1105000000000004e-05 2021-01-15T19:08:38.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:38.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196812+0000", 2021-01-15T19:08:38.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011315451000000001 2021-01-15T19:08:38.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208128+0000", 2021-01-15T19:08:38.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010873 2021-01-15T19:08:38.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:38.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208237+0000", 2021-01-15T19:08:38.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058576419999999997 2021-01-15T19:08:38.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214094+0000", 2021-01-15T19:08:38.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119562 2021-01-15T19:08:38.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214214+0000", 2021-01-15T19:08:38.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012588000000000001 2021-01-15T19:08:38.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214340+0000", 2021-01-15T19:08:38.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001 2021-01-15T19:08:38.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.340 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:323 3.1s0 3:80f33866:::smithi00140323-14:head [delete] snapc 11=[11,10] ondisk+write+peerstat_old+known_if_redirected e41)", 2021-01-15T19:08:38.341 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:38.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9288163229999995, 2021-01-15T19:08:38.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061698250000000003, 2021-01-15T19:08:38.342 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.342 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:38.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.343 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 323 2021-01-15T19:08:38.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.343 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:38.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:38.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6786999999999999e-05 2021-01-15T19:08:38.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196345+0000", 2021-01-15T19:08:38.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6189999999999996e-06 2021-01-15T19:08:38.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196351+0000", 2021-01-15T19:08:38.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1976e-05 2021-01-15T19:08:38.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196363+0000", 2021-01-15T19:08:38.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9249999999999995e-06 2021-01-15T19:08:38.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196371+0000", 2021-01-15T19:08:38.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047688599999999999 2021-01-15T19:08:38.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196848+0000", 2021-01-15T19:08:38.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010886 2021-01-15T19:08:38.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:38.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196957+0000", 2021-01-15T19:08:38.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087102499999999999 2021-01-15T19:08:38.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197828+0000", 2021-01-15T19:08:38.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022233589999999998 2021-01-15T19:08:38.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.200051+0000", 2021-01-15T19:08:38.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023389800000000001 2021-01-15T19:08:38.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:38.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202390+0000", 2021-01-15T19:08:38.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7408000000000001e-05 2021-01-15T19:08:38.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202478+0000", 2021-01-15T19:08:38.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061698250000000003 2021-01-15T19:08:38.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.357 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.358 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.358 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=127, reqid=client.4418.0:362, at_version=45'23, trim_to=0'0, roll_forward_to=39'21))", 2021-01-15T19:08:38.359 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.188586+0000", 2021-01-15T19:08:38.359 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9365376239999996, 2021-01-15T19:08:38.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050624770000000001, 2021-01-15T19:08:38.359 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.359 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188586+0000", 2021-01-15T19:08:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188586+0000", 2021-01-15T19:08:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018245800000000001 2021-01-15T19:08:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188769+0000", 2021-01-15T19:08:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0593000000000003e-05 2021-01-15T19:08:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188800+0000", 2021-01-15T19:08:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3529000000000001e-05 2021-01-15T19:08:38.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188823+0000", 2021-01-15T19:08:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0800000000000006e-06 2021-01-15T19:08:38.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188831+0000", 2021-01-15T19:08:38.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8584e-05 2021-01-15T19:08:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188890+0000", 2021-01-15T19:08:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6521e-05 2021-01-15T19:08:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.188906+0000", 2021-01-15T19:08:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046927840000000002 2021-01-15T19:08:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.193599+0000", 2021-01-15T19:08:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9928e-05 2021-01-15T19:08:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.193649+0000", 2021-01-15T19:08:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050624770000000001 2021-01-15T19:08:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.371 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.372 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=129, reqid=client.4418.0:364, at_version=45'25, trim_to=0'0, roll_forward_to=39'21))", 2021-01-15T19:08:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9283241069999999, 2021-01-15T19:08:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008030611, 2021-01-15T19:08:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015823399999999999 2021-01-15T19:08:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196958+0000", 2021-01-15T19:08:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1186999999999999e-05 2021-01-15T19:08:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196979+0000", 2021-01-15T19:08:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5041999999999999e-05 2021-01-15T19:08:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196994+0000", 2021-01-15T19:08:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7460000000000003e-06 2021-01-15T19:08:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197001+0000", 2021-01-15T19:08:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021975599999999999 2021-01-15T19:08:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197221+0000", 2021-01-15T19:08:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2945999999999999e-05 2021-01-15T19:08:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197234+0000", 2021-01-15T19:08:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075394920000000001 2021-01-15T19:08:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204773+0000", 2021-01-15T19:08:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7207999999999998e-05 2021-01-15T19:08:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204831+0000", 2021-01-15T19:08:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008030611 2021-01-15T19:08:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s1 45/21 ECSubWrite(tid=35, reqid=client.4418.0:374, at_version=45'10, trim_to=0'0, roll_forward_to=40'8))", 2021-01-15T19:08:38.386 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198660+0000", 2021-01-15T19:08:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9264636470000003, 2021-01-15T19:08:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005233054, 2021-01-15T19:08:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198660+0000", 2021-01-15T19:08:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198660+0000", 2021-01-15T19:08:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5329000000000001e-05 2021-01-15T19:08:38.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198686+0000", 2021-01-15T19:08:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4364e-05 2021-01-15T19:08:38.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198700+0000", 2021-01-15T19:08:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:08:38.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198709+0000", 2021-01-15T19:08:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5260000000000004e-06 2021-01-15T19:08:38.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198713+0000", 2021-01-15T19:08:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2893999999999999e-05 2021-01-15T19:08:38.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198766+0000", 2021-01-15T19:08:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2265e-05 2021-01-15T19:08:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198778+0000", 2021-01-15T19:08:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050576299999999996 2021-01-15T19:08:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.203836+0000", 2021-01-15T19:08:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7627000000000002e-05 2021-01-15T19:08:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.203894+0000", 2021-01-15T19:08:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005233054 2021-01-15T19:08:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s1 45/21 ECSubWrite(tid=37, reqid=client.4418.0:375, at_version=45'11, trim_to=0'0, roll_forward_to=40'8))", 2021-01-15T19:08:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198747+0000", 2021-01-15T19:08:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9263769169999998, 2021-01-15T19:08:38.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054298929999999999, 2021-01-15T19:08:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198747+0000", 2021-01-15T19:08:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198747+0000", 2021-01-15T19:08:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7022e-05 2021-01-15T19:08:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198774+0000", 2021-01-15T19:08:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0645999999999999e-05 2021-01-15T19:08:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198785+0000", 2021-01-15T19:08:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5989999999999996e-06 2021-01-15T19:08:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198792+0000", 2021-01-15T19:08:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.965e-06 2021-01-15T19:08:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198795+0000", 2021-01-15T19:08:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077181099999999996 2021-01-15T19:08:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.199567+0000", 2021-01-15T19:08:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3137e-05 2021-01-15T19:08:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.199580+0000", 2021-01-15T19:08:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045474110000000003 2021-01-15T19:08:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204128+0000", 2021-01-15T19:08:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9302000000000003e-05 2021-01-15T19:08:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204177+0000", 2021-01-15T19:08:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054298929999999999 2021-01-15T19:08:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=130, reqid=client.4418.0:365, at_version=45'26, trim_to=0'0, roll_forward_to=45'23))", 2021-01-15T19:08:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9262755409999999, 2021-01-15T19:08:38.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060228970000000001, 2021-01-15T19:08:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7685e-05 2021-01-15T19:08:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198876+0000", 2021-01-15T19:08:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2933e-05 2021-01-15T19:08:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198889+0000", 2021-01-15T19:08:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5310000000000007e-06 2021-01-15T19:08:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198899+0000", 2021-01-15T19:08:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:08:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198904+0000", 2021-01-15T19:08:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7002999999999997e-05 2021-01-15T19:08:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198951+0000", 2021-01-15T19:08:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2021-01-15T19:08:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198963+0000", 2021-01-15T19:08:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058719660000000002 2021-01-15T19:08:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204835+0000", 2021-01-15T19:08:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6568e-05 2021-01-15T19:08:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204871+0000", 2021-01-15T19:08:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060228970000000001 2021-01-15T19:08:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.423 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.423 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:38.425 INFO:tasks.rados.rados.0.smithi001.stdout:158: done (3 left) 2021-01-15T19:08:38.425 INFO:tasks.rados.rados.0.smithi001.stdout:159: snap_create 2021-01-15T19:08:38.425 INFO:tasks.rados.rados.0.smithi001.stdout:155: finishing write tid 2 to smithi00140323-25 2021-01-15T19:08:38.426 INFO:tasks.rados.rados.0.smithi001.stdout:155: finishing write tid 3 to smithi00140323-25 2021-01-15T19:08:38.426 INFO:tasks.rados.rados.0.smithi001.stdout:156: finishing write tid 1 to smithi00140323-3 2021-01-15T19:08:38.426 INFO:tasks.rados.rados.0.smithi001.stdout:155: finishing write tid 4 to smithi00140323-25 2021-01-15T19:08:38.426 INFO:tasks.rados.rados.0.smithi001.stdout:155: finishing write tid 5 to smithi00140323-25 2021-01-15T19:08:38.427 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 28 (ObjNum 81 snap 14 seq_num 81) dirty exists 2021-01-15T19:08:38.427 INFO:tasks.rados.rados.0.smithi001.stdout:155: left oid 25 (ObjNum 81 snap 14 seq_num 81) 2021-01-15T19:08:38.427 INFO:tasks.rados.rados.0.smithi001.stdout:156: finishing write tid 2 to smithi00140323-3 2021-01-15T19:08:38.427 INFO:tasks.rados.rados.0.smithi001.stdout:156: finishing write tid 3 to smithi00140323-3 2021-01-15T19:08:38.427 INFO:tasks.rados.rados.0.smithi001.stdout:156: finishing write tid 4 to smithi00140323-3 2021-01-15T19:08:38.428 INFO:tasks.rados.rados.0.smithi001.stdout:157: finishing write tid 1 to smithi00140323-30 2021-01-15T19:08:38.428 INFO:tasks.rados.rados.0.smithi001.stdout:156: finishing write tid 5 to smithi00140323-3 2021-01-15T19:08:38.428 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 48 (ObjNum 82 snap 14 seq_num 82) dirty exists 2021-01-15T19:08:38.428 INFO:tasks.rados.rados.0.smithi001.stdout:156: left oid 3 (ObjNum 82 snap 14 seq_num 82) 2021-01-15T19:08:38.428 INFO:tasks.rados.rados.0.smithi001.stdout:157: finishing write tid 2 to smithi00140323-30 2021-01-15T19:08:38.429 INFO:tasks.rados.rados.0.smithi001.stdout:157: finishing write tid 3 to smithi00140323-30 2021-01-15T19:08:38.429 INFO:tasks.rados.rados.0.smithi001.stdout:157: finishing write tid 4 to smithi00140323-30 2021-01-15T19:08:38.429 INFO:tasks.rados.rados.0.smithi001.stdout:157: finishing write tid 5 to smithi00140323-30 2021-01-15T19:08:38.429 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 30 v 9 (ObjNum 83 snap 14 seq_num 83) dirty exists 2021-01-15T19:08:38.429 INFO:tasks.rados.rados.0.smithi001.stdout:157: left oid 30 (ObjNum 83 snap 14 seq_num 83) 2021-01-15T19:08:38.430 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 3 pg_num to 26 2021-01-15T19:08:38.430 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:38.422+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:38.431 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:38.525 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:38.524+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:38.594 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:38.595 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:38.604 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:38.626 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:38.624+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:38.731 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:38.724+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:38.734 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:38.735 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:38.744 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:38.869 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:38.828+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:38.915 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=84, reqid=client.4418.0:311, at_version=41'12, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7166643280000002, 2021-01-15T19:08:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049070149999999998, 2021-01-15T19:08:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196260+0000", 2021-01-15T19:08:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065051599999999996 2021-01-15T19:08:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196911+0000", 2021-01-15T19:08:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2234000000000001e-05 2021-01-15T19:08:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196933+0000", 2021-01-15T19:08:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4019000000000001e-05 2021-01-15T19:08:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196947+0000", 2021-01-15T19:08:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1910000000000003e-06 2021-01-15T19:08:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196954+0000", 2021-01-15T19:08:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7198999999999999e-05 2021-01-15T19:08:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197012+0000", 2021-01-15T19:08:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5761999999999999e-05 2021-01-15T19:08:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197027+0000", 2021-01-15T19:08:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004104697 2021-01-15T19:08:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.201132+0000", 2021-01-15T19:08:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5397e-05 2021-01-15T19:08:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.201167+0000", 2021-01-15T19:08:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049070149999999998 2021-01-15T19:08:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=87, reqid=client.4418.0:313, at_version=41'14, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7083139999999997, 2021-01-15T19:08:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998, 2021-01-15T19:08:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0902000000000002e-05 2021-01-15T19:08:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204662+0000", 2021-01-15T19:08:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8193e-05 2021-01-15T19:08:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204680+0000", 2021-01-15T19:08:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958e-05 2021-01-15T19:08:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204698+0000", 2021-01-15T19:08:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2021-01-15T19:08:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204706+0000", 2021-01-15T19:08:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2994000000000002e-05 2021-01-15T19:08:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204749+0000", 2021-01-15T19:08:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491999999999999e-05 2021-01-15T19:08:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204760+0000", 2021-01-15T19:08:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056169770000000004 2021-01-15T19:08:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210377+0000", 2021-01-15T19:08:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8433999999999997e-05 2021-01-15T19:08:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210425+0000", 2021-01-15T19:08:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998 2021-01-15T19:08:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 41/21 ECSubWrite(tid=115, reqid=client.4418.0:324, at_version=41'23, trim_to=0'0, roll_forward_to=41'21))", 2021-01-15T19:08:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7006941409999996, 2021-01-15T19:08:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00487003, 2021-01-15T19:08:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212231+0000", 2021-01-15T19:08:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3484000000000002e-05 2021-01-15T19:08:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212314+0000", 2021-01-15T19:08:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7509000000000001e-05 2021-01-15T19:08:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212332+0000", 2021-01-15T19:08:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1646e-05 2021-01-15T19:08:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212343+0000", 2021-01-15T19:08:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:08:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212349+0000", 2021-01-15T19:08:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3396999999999997e-05 2021-01-15T19:08:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212402+0000", 2021-01-15T19:08:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3617e-05 2021-01-15T19:08:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.212416+0000", 2021-01-15T19:08:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046101270000000003 2021-01-15T19:08:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217026+0000", 2021-01-15T19:08:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4484e-05 2021-01-15T19:08:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217101+0000", 2021-01-15T19:08:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00487003 2021-01-15T19:08:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.6893255749999998, 2021-01-15T19:08:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=44, reqid=client.4418.0:334, at_version=41'25, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.6876313580000009, 2021-01-15T19:08:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001, 2021-01-15T19:08:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9133000000000004e-05 2021-01-15T19:08:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225363+0000", 2021-01-15T19:08:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2021-01-15T19:08:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225376+0000", 2021-01-15T19:08:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0949e-05 2021-01-15T19:08:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225387+0000", 2021-01-15T19:08:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7710000000000001e-06 2021-01-15T19:08:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225394+0000", 2021-01-15T19:08:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5732999999999997e-05 2021-01-15T19:08:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225440+0000", 2021-01-15T19:08:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5984000000000001e-05 2021-01-15T19:08:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225456+0000", 2021-01-15T19:08:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059787429999999999 2021-01-15T19:08:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231434+0000", 2021-01-15T19:08:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2653000000000001e-05 2021-01-15T19:08:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231457+0000", 2021-01-15T19:08:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001 2021-01-15T19:08:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=45, reqid=client.4418.0:335, at_version=41'26, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.685944632, 2021-01-15T19:08:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047473949999999997, 2021-01-15T19:08:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226980+0000", 2021-01-15T19:08:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0362000000000003e-05 2021-01-15T19:08:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227031+0000", 2021-01-15T19:08:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9890000000000001e-05 2021-01-15T19:08:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227050+0000", 2021-01-15T19:08:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7663e-05 2021-01-15T19:08:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227068+0000", 2021-01-15T19:08:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4250000000000003e-06 2021-01-15T19:08:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227075+0000", 2021-01-15T19:08:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8297999999999997e-05 2021-01-15T19:08:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227123+0000", 2021-01-15T19:08:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0387000000000001e-05 2021-01-15T19:08:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227133+0000", 2021-01-15T19:08:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004554598 2021-01-15T19:08:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231688+0000", 2021-01-15T19:08:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9771999999999998e-05 2021-01-15T19:08:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231728+0000", 2021-01-15T19:08:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047473949999999997 2021-01-15T19:08:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.990 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:338 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:38.991 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179693+0000", 2021-01-15T19:08:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.7332314560000004, 2021-01-15T19:08:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048277429999999998, 2021-01-15T19:08:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:38.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 338 2021-01-15T19:08:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.993 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179693+0000", 2021-01-15T19:08:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:38.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179693+0000", 2021-01-15T19:08:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5828999999999999e-05 2021-01-15T19:08:38.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179729+0000", 2021-01-15T19:08:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0650000000000001e-06 2021-01-15T19:08:38.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179736+0000", 2021-01-15T19:08:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2021-01-15T19:08:38.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179758+0000", 2021-01-15T19:08:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2945000000000001e-05 2021-01-15T19:08:38.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179771+0000", 2021-01-15T19:08:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4350000000000004e-05 2021-01-15T19:08:38.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179855+0000", 2021-01-15T19:08:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3669000000000001e-05 2021-01-15T19:08:39.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179869+0000", 2021-01-15T19:08:39.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019447589999999999 2021-01-15T19:08:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.181814+0000", 2021-01-15T19:08:39.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015279099999999999 2021-01-15T19:08:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.181966+0000", 2021-01-15T19:08:39.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025547259999999998 2021-01-15T19:08:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.184521+0000", 2021-01-15T19:08:39.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048277429999999998 2021-01-15T19:08:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:339 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179837+0000", 2021-01-15T19:08:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.7330879229999994, 2021-01-15T19:08:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052755500000000004, 2021-01-15T19:08:39.006 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.007 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 339 2021-01-15T19:08:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179837+0000", 2021-01-15T19:08:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179837+0000", 2021-01-15T19:08:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6734999999999998e-05 2021-01-15T19:08:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179894+0000", 2021-01-15T19:08:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8279999999999998e-06 2021-01-15T19:08:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179903+0000", 2021-01-15T19:08:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4875000000000001e-05 2021-01-15T19:08:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179918+0000", 2021-01-15T19:08:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6220000000000003e-06 2021-01-15T19:08:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179925+0000", 2021-01-15T19:08:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2964000000000003e-05 2021-01-15T19:08:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179978+0000", 2021-01-15T19:08:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3149999999999998e-06 2021-01-15T19:08:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:39.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179987+0000", 2021-01-15T19:08:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029067009999999998 2021-01-15T19:08:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.182894+0000", 2021-01-15T19:08:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8940000000000001e-05 2021-01-15T19:08:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:39.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.182963+0000", 2021-01-15T19:08:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00214957 2021-01-15T19:08:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.185112+0000", 2021-01-15T19:08:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052755500000000004 2021-01-15T19:08:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:340 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.7329510789999993, 2021-01-15T19:08:39.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056637780000000004, 2021-01-15T19:08:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:39.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 340 2021-01-15T19:08:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6628000000000001e-05 2021-01-15T19:08:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180030+0000", 2021-01-15T19:08:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:08:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180038+0000", 2021-01-15T19:08:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3541e-05 2021-01-15T19:08:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180051+0000", 2021-01-15T19:08:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:08:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180056+0000", 2021-01-15T19:08:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3109999999999998e-05 2021-01-15T19:08:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180110+0000", 2021-01-15T19:08:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9489999999999996e-06 2021-01-15T19:08:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180118+0000", 2021-01-15T19:08:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033250100000000002 2021-01-15T19:08:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.183443+0000", 2021-01-15T19:08:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2630000000000002e-05 2021-01-15T19:08:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.183505+0000", 2021-01-15T19:08:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021323370000000002 2021-01-15T19:08:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.185638+0000", 2021-01-15T19:08:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056637780000000004 2021-01-15T19:08:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.033 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 45/21 ECSubWrite(tid=55, reqid=client.4418.0:372, at_version=45'35, trim_to=0'0, roll_forward_to=44'31))", 2021-01-15T19:08:39.034 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.206009+0000", 2021-01-15T19:08:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7069153850000003, 2021-01-15T19:08:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048428999999999998, 2021-01-15T19:08:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.035 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206009+0000", 2021-01-15T19:08:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206009+0000", 2021-01-15T19:08:39.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197584 2021-01-15T19:08:39.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206207+0000", 2021-01-15T19:08:39.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7834000000000002e-05 2021-01-15T19:08:39.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206235+0000", 2021-01-15T19:08:39.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059999999999999e-05 2021-01-15T19:08:39.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206252+0000", 2021-01-15T19:08:39.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0770000000000002e-06 2021-01-15T19:08:39.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206259+0000", 2021-01-15T19:08:39.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6487999999999999e-05 2021-01-15T19:08:39.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206305+0000", 2021-01-15T19:08:39.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2275e-05 2021-01-15T19:08:39.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206318+0000", 2021-01-15T19:08:39.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044787849999999999 2021-01-15T19:08:39.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.210797+0000", 2021-01-15T19:08:39.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5797000000000001e-05 2021-01-15T19:08:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.210852+0000", 2021-01-15T19:08:39.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048428999999999998 2021-01-15T19:08:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.047 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/21 ECSubWrite(tid=57, reqid=client.4418.0:379, at_version=47'36, trim_to=0'0, roll_forward_to=45'35))", 2021-01-15T19:08:39.047 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.215710+0000", 2021-01-15T19:08:39.047 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6972150780000002, 2021-01-15T19:08:39.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046306519999999999, 2021-01-15T19:08:39.047 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.048 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.048 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215710+0000", 2021-01-15T19:08:39.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215710+0000", 2021-01-15T19:08:39.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165465 2021-01-15T19:08:39.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215875+0000", 2021-01-15T19:08:39.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8351000000000001e-05 2021-01-15T19:08:39.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215894+0000", 2021-01-15T19:08:39.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2537e-05 2021-01-15T19:08:39.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215906+0000", 2021-01-15T19:08:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:08:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215913+0000", 2021-01-15T19:08:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9874000000000003e-05 2021-01-15T19:08:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215973+0000", 2021-01-15T19:08:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5590000000000002e-05 2021-01-15T19:08:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215989+0000", 2021-01-15T19:08:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043006720000000002 2021-01-15T19:08:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.220290+0000", 2021-01-15T19:08:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0773000000000003e-05 2021-01-15T19:08:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.220340+0000", 2021-01-15T19:08:39.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046306519999999999 2021-01-15T19:08:39.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.058 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/21 ECSubWrite(tid=58, reqid=client.4418.0:380, at_version=47'37, trim_to=0'0, roll_forward_to=45'35))", 2021-01-15T19:08:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.220838+0000", 2021-01-15T19:08:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6920872930000002, 2021-01-15T19:08:39.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050639250000000004, 2021-01-15T19:08:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.220838+0000", 2021-01-15T19:08:39.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.220838+0000", 2021-01-15T19:08:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021222799999999999 2021-01-15T19:08:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.221050+0000", 2021-01-15T19:08:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7621999999999998e-05 2021-01-15T19:08:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.221067+0000", 2021-01-15T19:08:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.345e-05 2021-01-15T19:08:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.221081+0000", 2021-01-15T19:08:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.674e-06 2021-01-15T19:08:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.221087+0000", 2021-01-15T19:08:39.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3529000000000002e-05 2021-01-15T19:08:39.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.221141+0000", 2021-01-15T19:08:39.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:08:39.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.221156+0000", 2021-01-15T19:08:39.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046967279999999998 2021-01-15T19:08:39.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.225852+0000", 2021-01-15T19:08:39.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.905e-05 2021-01-15T19:08:39.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.225901+0000", 2021-01-15T19:08:39.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050639250000000004 2021-01-15T19:08:39.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.071 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.072 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/21 ECSubWrite(tid=61, reqid=client.4418.0:382, at_version=0'0, trim_to=0'0, roll_forward_to=47'39))", 2021-01-15T19:08:39.073 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.226955+0000", 2021-01-15T19:08:39.073 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6859702890000001, 2021-01-15T19:08:39.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044897749999999997, 2021-01-15T19:08:39.074 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.074 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.074 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.226955+0000", 2021-01-15T19:08:39.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.226955+0000", 2021-01-15T19:08:39.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5428000000000001e-05 2021-01-15T19:08:39.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.226980+0000", 2021-01-15T19:08:39.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9000000000000001e-06 2021-01-15T19:08:39.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.226990+0000", 2021-01-15T19:08:39.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1114000000000001e-05 2021-01-15T19:08:39.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227001+0000", 2021-01-15T19:08:39.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4450000000000004e-06 2021-01-15T19:08:39.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227006+0000", 2021-01-15T19:08:39.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0288999999999999e-05 2021-01-15T19:08:39.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227057+0000", 2021-01-15T19:08:39.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3974999999999999e-05 2021-01-15T19:08:39.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227071+0000", 2021-01-15T19:08:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043443450000000003 2021-01-15T19:08:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231415+0000", 2021-01-15T19:08:39.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9278999999999999e-05 2021-01-15T19:08:39.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231444+0000", 2021-01-15T19:08:39.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044897749999999997 2021-01-15T19:08:39.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.086 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.086 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 47/21 ECSubWrite(tid=101, reqid=client.4418.0:384, at_version=47'23, trim_to=0'0, roll_forward_to=44'21))", 2021-01-15T19:08:39.087 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:39.087 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.67180426, 2021-01-15T19:08:39.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002, 2021-01-15T19:08:39.087 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.087 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:39.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:39.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029587999999999999 2021-01-15T19:08:39.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241416+0000", 2021-01-15T19:08:39.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6582000000000001e-05 2021-01-15T19:08:39.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241433+0000", 2021-01-15T19:08:39.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2021-01-15T19:08:39.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241444+0000", 2021-01-15T19:08:39.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.63e-06 2021-01-15T19:08:39.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241451+0000", 2021-01-15T19:08:39.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6316999999999997e-05 2021-01-15T19:08:39.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241497+0000", 2021-01-15T19:08:39.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:39.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241506+0000", 2021-01-15T19:08:39.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058582230000000001 2021-01-15T19:08:39.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247365+0000", 2021-01-15T19:08:39.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6705e-05 2021-01-15T19:08:39.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247421+0000", 2021-01-15T19:08:39.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002 2021-01-15T19:08:39.098 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.098 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.098 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.099 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 47/21 ECSubWrite(tid=102, reqid=client.4418.0:385, at_version=47'24, trim_to=0'0, roll_forward_to=44'21))", 2021-01-15T19:08:39.099 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.248021+0000", 2021-01-15T19:08:39.099 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6649038269999998, 2021-01-15T19:08:39.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045089329999999997, 2021-01-15T19:08:39.099 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248021+0000", 2021-01-15T19:08:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248021+0000", 2021-01-15T19:08:39.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031588900000000002 2021-01-15T19:08:39.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248337+0000", 2021-01-15T19:08:39.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0290000000000001e-05 2021-01-15T19:08:39.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248357+0000", 2021-01-15T19:08:39.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3988000000000001e-05 2021-01-15T19:08:39.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248371+0000", 2021-01-15T19:08:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2349999999999999e-06 2021-01-15T19:08:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248379+0000", 2021-01-15T19:08:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1996000000000002e-05 2021-01-15T19:08:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248421+0000", 2021-01-15T19:08:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8439999999999998e-06 2021-01-15T19:08:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.248429+0000", 2021-01-15T19:08:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040558749999999996 2021-01-15T19:08:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.252485+0000", 2021-01-15T19:08:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4815999999999999e-05 2021-01-15T19:08:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.252530+0000", 2021-01-15T19:08:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045089329999999997 2021-01-15T19:08:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=50'50, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.73158593900000002, 2021-01-15T19:08:39.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996, 2021-01-15T19:08:39.112 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.112 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.112 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:39.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:39.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9509999999999999e-05 2021-01-15T19:08:39.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181388+0000", 2021-01-15T19:08:39.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.141e-05 2021-01-15T19:08:39.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181410+0000", 2021-01-15T19:08:39.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5201000000000001e-05 2021-01-15T19:08:39.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181425+0000", 2021-01-15T19:08:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5440000000000004e-06 2021-01-15T19:08:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181432+0000", 2021-01-15T19:08:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029235099999999999 2021-01-15T19:08:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181724+0000", 2021-01-15T19:08:39.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1324e-05 2021-01-15T19:08:39.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181735+0000", 2021-01-15T19:08:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005959865 2021-01-15T19:08:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187695+0000", 2021-01-15T19:08:39.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4237999999999999e-05 2021-01-15T19:08:39.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187739+0000", 2021-01-15T19:08:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996 2021-01-15T19:08:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.122 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=73, reqid=osd.5.0:73, at_version=50'52, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:39.123 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.73117225799999996, 2021-01-15T19:08:39.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003, 2021-01-15T19:08:39.124 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.124 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.124 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7224000000000001e-05 2021-01-15T19:08:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181800+0000", 2021-01-15T19:08:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5681999999999999e-05 2021-01-15T19:08:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181825+0000", 2021-01-15T19:08:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5474000000000001e-05 2021-01-15T19:08:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181841+0000", 2021-01-15T19:08:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8179999999999993e-06 2021-01-15T19:08:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181850+0000", 2021-01-15T19:08:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150703 2021-01-15T19:08:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182000+0000", 2021-01-15T19:08:39.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2021-01-15T19:08:39.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182010+0000", 2021-01-15T19:08:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066268780000000001 2021-01-15T19:08:39.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188637+0000", 2021-01-15T19:08:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6959000000000002e-05 2021-01-15T19:08:39.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188684+0000", 2021-01-15T19:08:39.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003 2021-01-15T19:08:39.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.135 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=152, reqid=osd.4.0:152, at_version=50'30, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.72815560499999998, 2021-01-15T19:08:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998, 2021-01-15T19:08:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.136 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:39.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1204e-05 2021-01-15T19:08:39.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184820+0000", 2021-01-15T19:08:39.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1126000000000001e-05 2021-01-15T19:08:39.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184842+0000", 2021-01-15T19:08:39.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5197999999999999e-05 2021-01-15T19:08:39.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184857+0000", 2021-01-15T19:08:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.336e-06 2021-01-15T19:08:39.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184863+0000", 2021-01-15T19:08:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2200000000000002e-05 2021-01-15T19:08:39.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184915+0000", 2021-01-15T19:08:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999e-05 2021-01-15T19:08:39.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184927+0000", 2021-01-15T19:08:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065179089999999997 2021-01-15T19:08:39.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191445+0000", 2021-01-15T19:08:39.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2212999999999997e-05 2021-01-15T19:08:39.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191487+0000", 2021-01-15T19:08:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998 2021-01-15T19:08:39.146 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.147 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=153, reqid=osd.4.0:153, at_version=50'32, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:39.148 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:39.148 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.728013256, 2021-01-15T19:08:39.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999, 2021-01-15T19:08:39.148 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.148 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.149 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:39.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:39.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0540000000000001e-05 2021-01-15T19:08:39.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184962+0000", 2021-01-15T19:08:39.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1452999999999999e-05 2021-01-15T19:08:39.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184984+0000", 2021-01-15T19:08:39.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4532000000000001e-05 2021-01-15T19:08:39.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184998+0000", 2021-01-15T19:08:39.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9069999999999998e-06 2021-01-15T19:08:39.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.185003+0000", 2021-01-15T19:08:39.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027679219999999999 2021-01-15T19:08:39.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187771+0000", 2021-01-15T19:08:39.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1089e-05 2021-01-15T19:08:39.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187782+0000", 2021-01-15T19:08:39.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004853792 2021-01-15T19:08:39.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192636+0000", 2021-01-15T19:08:39.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0936000000000003e-05 2021-01-15T19:08:39.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192667+0000", 2021-01-15T19:08:39.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999 2021-01-15T19:08:39.159 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.159 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.159 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.160 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 50/21 ECSubWrite(tid=114, reqid=osd.3.0:114, at_version=50'32, trim_to=0'0, roll_forward_to=48'28))", 2021-01-15T19:08:39.160 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.511994+0000", 2021-01-15T19:08:39.160 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.400930642, 2021-01-15T19:08:39.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004117908, 2021-01-15T19:08:39.160 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:39.161 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:39.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:39.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:39.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.511994+0000", 2021-01-15T19:08:39.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:39.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:39.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.511994+0000", 2021-01-15T19:08:39.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9312e-05 2021-01-15T19:08:39.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:39.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.512023+0000", 2021-01-15T19:08:39.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8202e-05 2021-01-15T19:08:39.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:39.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.512042+0000", 2021-01-15T19:08:39.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1194e-05 2021-01-15T19:08:39.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:39.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.512053+0000", 2021-01-15T19:08:39.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4839999999999998e-06 2021-01-15T19:08:39.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.512060+0000", 2021-01-15T19:08:39.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9064999999999998e-05 2021-01-15T19:08:39.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:39.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.512119+0000", 2021-01-15T19:08:39.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4335000000000001e-05 2021-01-15T19:08:39.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.512134+0000", 2021-01-15T19:08:39.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003940842 2021-01-15T19:08:39.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.516075+0000", 2021-01-15T19:08:39.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7474000000000001e-05 2021-01-15T19:08:39.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:39.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:39.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:39.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.516112+0000", 2021-01-15T19:08:39.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004117908 2021-01-15T19:08:39.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:39.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:39.172 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:39.173 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:39.174 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:39.172+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:39.181 INFO:tasks.rados.rados.0.smithi001.stdout:155: done (3 left) 2021-01-15T19:08:39.182 INFO:tasks.rados.rados.0.smithi001.stdout:156: done (2 left) 2021-01-15T19:08:39.182 INFO:tasks.rados.rados.0.smithi001.stdout:157: done (1 left) 2021-01-15T19:08:39.182 INFO:tasks.rados.rados.0.smithi001.stdout:159: done (0 left) 2021-01-15T19:08:39.182 INFO:tasks.rados.rados.0.smithi001.stdout:160: delete oid 3 current snap is 15 2021-01-15T19:08:39.217 INFO:tasks.rados.rados.0.smithi001.stdout:160: done (0 left) 2021-01-15T19:08:39.218 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 38 current snap is 15 2021-01-15T19:08:39.218 INFO:tasks.rados.rados.0.smithi001.stdout:161: seq_num 85 ranges {622592=630784,1253376=720896,1974272=81920} 2021-01-15T19:08:39.224 INFO:tasks.rados.rados.0.smithi001.stdout:161: writing smithi00140323-38 from 622592 to 1253376 tid 1 2021-01-15T19:08:39.228 INFO:tasks.rados.rados.0.smithi001.stdout:161: writing smithi00140323-38 from 1253376 to 1974272 tid 2 2021-01-15T19:08:39.229 INFO:tasks.rados.rados.0.smithi001.stdout:161: writing smithi00140323-38 from 1974272 to 2056192 tid 3 2021-01-15T19:08:39.229 INFO:tasks.rados.rados.0.smithi001.stdout:162: copy_from oid 35 from oid 5 current snap is 15 2021-01-15T19:08:39.230 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 47 current snap is 15 2021-01-15T19:08:39.230 INFO:tasks.rados.rados.0.smithi001.stdout:163: seq_num 86 ranges {1900544=647168,2547712=606208,3153920=196608} 2021-01-15T19:08:39.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:08:39.244 INFO:tasks.rados.rados.0.smithi001.stdout:163: writing smithi00140323-47 from 1900544 to 2547712 tid 1 2021-01-15T19:08:39.247 INFO:tasks.rados.rados.0.smithi001.stdout:163: writing smithi00140323-47 from 2547712 to 3153920 tid 2 2021-01-15T19:08:39.249 INFO:tasks.rados.rados.0.smithi001.stdout:163: writing smithi00140323-47 from 3153920 to 3350528 tid 3 2021-01-15T19:08:39.249 INFO:tasks.rados.rados.0.smithi001.stdout:164: copy_from oid 2 from oid 22 current snap is 15 2021-01-15T19:08:39.250 INFO:tasks.rados.rados.0.smithi001.stdout:161: finishing write tid 1 to smithi00140323-38 2021-01-15T19:08:39.250 INFO:tasks.rados.rados.0.smithi001.stdout:161: finishing write tid 2 to smithi00140323-38 2021-01-15T19:08:39.250 INFO:tasks.rados.rados.0.smithi001.stdout:161: finishing write tid 3 to smithi00140323-38 2021-01-15T19:08:39.250 INFO:tasks.rados.rados.0.smithi001.stdout:161: finishing write tid 4 to smithi00140323-38 2021-01-15T19:08:39.250 INFO:tasks.rados.rados.0.smithi001.stdout:162: finishing copy_from to smithi00140323-35 2021-01-15T19:08:39.251 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 4 (ObjNum 4 snap 0 seq_num 4) dirty exists 2021-01-15T19:08:39.251 INFO:tasks.rados.rados.0.smithi001.stdout:161: finishing write tid 5 to smithi00140323-38 2021-01-15T19:08:39.251 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 26 (ObjNum 85 snap 15 seq_num 85) dirty exists 2021-01-15T19:08:39.251 INFO:tasks.rados.rados.0.smithi001.stdout:161: left oid 38 (ObjNum 85 snap 15 seq_num 85) 2021-01-15T19:08:39.251 INFO:tasks.rados.rados.0.smithi001.stdout:162: finishing copy_from racing read to smithi00140323-35 2021-01-15T19:08:39.252 INFO:tasks.rados.rados.0.smithi001.stdout:161: done (3 left) 2021-01-15T19:08:39.252 INFO:tasks.rados.rados.0.smithi001.stdout:162: done (2 left) 2021-01-15T19:08:39.252 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 33 current snap is 15 2021-01-15T19:08:39.252 INFO:tasks.rados.rados.0.smithi001.stdout:165: seq_num 87 ranges {974848=663552,1638400=491520,2129920=311296} 2021-01-15T19:08:39.258 INFO:tasks.rados.rados.0.smithi001.stdout:165: writing smithi00140323-33 from 974848 to 1638400 tid 1 2021-01-15T19:08:39.262 INFO:tasks.rados.rados.0.smithi001.stdout:165: writing smithi00140323-33 from 1638400 to 2129920 tid 2 2021-01-15T19:08:39.264 INFO:tasks.rados.rados.0.smithi001.stdout:165: writing smithi00140323-33 from 2129920 to 2441216 tid 3 2021-01-15T19:08:39.264 INFO:tasks.rados.rados.0.smithi001.stdout:166: read oid 30 snap -1 2021-01-15T19:08:39.265 INFO:tasks.rados.rados.0.smithi001.stdout:166: expect (ObjNum 83 snap 14 seq_num 83) 2021-01-15T19:08:39.265 INFO:tasks.rados.rados.0.smithi001.stdout:167: read oid 4 snap -1 2021-01-15T19:08:39.265 INFO:tasks.rados.rados.0.smithi001.stdout:167: expect (ObjNum 3 snap 0 seq_num 3) 2021-01-15T19:08:39.265 INFO:tasks.rados.rados.0.smithi001.stdout:168: rollback oid 31 current snap is 15 2021-01-15T19:08:39.266 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 31 to 10 2021-01-15T19:08:39.266 INFO:tasks.rados.rados.0.smithi001.stdout:169: copy_from oid 25 from oid 46 current snap is 15 2021-01-15T19:08:39.266 INFO:tasks.rados.rados.0.smithi001.stdout:163: finishing write tid 1 to smithi00140323-47 2021-01-15T19:08:39.266 INFO:tasks.rados.rados.0.smithi001.stdout:163: finishing write tid 2 to smithi00140323-47 2021-01-15T19:08:39.266 INFO:tasks.rados.rados.0.smithi001.stdout:163: finishing write tid 3 to smithi00140323-47 2021-01-15T19:08:39.267 INFO:tasks.rados.rados.0.smithi001.stdout:163: finishing write tid 4 to smithi00140323-47 2021-01-15T19:08:39.267 INFO:tasks.rados.rados.0.smithi001.stdout:163: finishing write tid 5 to smithi00140323-47 2021-01-15T19:08:39.267 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 61 (ObjNum 86 snap 15 seq_num 86) dirty exists 2021-01-15T19:08:39.267 INFO:tasks.rados.rados.0.smithi001.stdout:163: left oid 47 (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:08:39.268 INFO:tasks.rados.rados.0.smithi001.stdout:163: done (6 left) 2021-01-15T19:08:39.268 INFO:tasks.rados.rados.0.smithi001.stdout:170: snap_create 2021-01-15T19:08:39.273 INFO:tasks.rados.rados.0.smithi001.stdout:168: finishing rollback tid 0 to smithi00140323-31 2021-01-15T19:08:39.274 INFO:tasks.rados.rados.0.smithi001.stdout:168: finishing rollback tid 1 to smithi00140323-31 2021-01-15T19:08:39.275 INFO:tasks.rados.rados.0.smithi001.stdout:167: expect (ObjNum 3 snap 0 seq_num 3) 2021-01-15T19:08:39.275 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:39.271+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:39.277 INFO:tasks.rados.rados.0.smithi001.stdout:164: finishing copy_from to smithi00140323-2 2021-01-15T19:08:39.277 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2021-01-15T19:08:39.277 INFO:tasks.rados.rados.0.smithi001.stdout:164: finishing copy_from racing read to smithi00140323-2 2021-01-15T19:08:39.278 INFO:tasks.rados.rados.0.smithi001.stdout:168: finishing rollback tid 2 to smithi00140323-31 2021-01-15T19:08:39.278 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 20 (ObjNum 30 snap 0 seq_num 30) dirty exists 2021-01-15T19:08:39.280 INFO:tasks.rados.rados.0.smithi001.stdout:166: expect (ObjNum 83 snap 14 seq_num 83) 2021-01-15T19:08:39.340 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:39.342 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:39.345 INFO:tasks.rados.rados.0.smithi001.stdout:165: finishing write tid 1 to smithi00140323-33 2021-01-15T19:08:39.346 INFO:tasks.rados.rados.0.smithi001.stdout:165: finishing write tid 2 to smithi00140323-33 2021-01-15T19:08:39.346 INFO:tasks.rados.rados.0.smithi001.stdout:165: finishing write tid 3 to smithi00140323-33 2021-01-15T19:08:39.346 INFO:tasks.rados.rados.0.smithi001.stdout:165: finishing write tid 4 to smithi00140323-33 2021-01-15T19:08:39.346 INFO:tasks.rados.rados.0.smithi001.stdout:169: finishing copy_from to smithi00140323-25 2021-01-15T19:08:39.346 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 10 (ObjNum 45 snap 0 seq_num 45) dirty exists 2021-01-15T19:08:39.347 INFO:tasks.rados.rados.0.smithi001.stdout:169: finishing copy_from racing read to smithi00140323-25 2021-01-15T19:08:39.347 INFO:tasks.rados.rados.0.smithi001.stdout:165: finishing write tid 5 to smithi00140323-33 2021-01-15T19:08:39.347 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 43 (ObjNum 87 snap 15 seq_num 87) dirty exists 2021-01-15T19:08:39.347 INFO:tasks.rados.rados.0.smithi001.stdout:165: left oid 33 (ObjNum 87 snap 15 seq_num 87) 2021-01-15T19:08:39.351 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:39.374 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:39.373+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:39.475 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:39.473+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:39.524 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:39.525 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:39.525 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:39.525 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:39.525 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:39.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:39.576 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:39.574+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:39.668 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:39.669 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:08:39.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:08:39.669 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:08:39.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:154 3.fs0 3:f27a3046:::smithi00140323-1:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:39.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:39.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.428501601000001, 2021-01-15T19:08:39.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002, 2021-01-15T19:08:39.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.671 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 154 2021-01-15T19:08:39.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.672 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:39.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237566+0000", 2021-01-15T19:08:39.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4352999999999999e-05 2021-01-15T19:08:39.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237631+0000", 2021-01-15T19:08:39.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3304e-05 2021-01-15T19:08:39.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237644+0000", 2021-01-15T19:08:39.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1736e-05 2021-01-15T19:08:39.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237666+0000", 2021-01-15T19:08:39.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0607e-05 2021-01-15T19:08:39.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237676+0000", 2021-01-15T19:08:39.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3862e-05 2021-01-15T19:08:39.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237730+0000", 2021-01-15T19:08:39.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1849999999999998e-06 2021-01-15T19:08:39.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:39.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237737+0000", 2021-01-15T19:08:39.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014276250000000001 2021-01-15T19:08:39.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239165+0000", 2021-01-15T19:08:39.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7447000000000007e-05 2021-01-15T19:08:39.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239232+0000", 2021-01-15T19:08:39.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061349480000000003 2021-01-15T19:08:39.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245367+0000", 2021-01-15T19:08:39.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0095382000000000002 2021-01-15T19:08:39.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254906+0000", 2021-01-15T19:08:39.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113465 2021-01-15T19:08:39.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255019+0000", 2021-01-15T19:08:39.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000454659 2021-01-15T19:08:39.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255474+0000", 2021-01-15T19:08:39.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017907391000000002 2021-01-15T19:08:39.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.686 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.687 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:155 3.fs0 3:f27a3046:::smithi00140323-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:39.687 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:39.688 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.428340493, 2021-01-15T19:08:39.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999, 2021-01-15T19:08:39.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 155 2021-01-15T19:08:39.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.689 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:39.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237727+0000", 2021-01-15T19:08:39.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9607999999999998e-05 2021-01-15T19:08:39.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237767+0000", 2021-01-15T19:08:39.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3810000000000001e-06 2021-01-15T19:08:39.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237771+0000", 2021-01-15T19:08:39.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5639999999999999e-06 2021-01-15T19:08:39.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237779+0000", 2021-01-15T19:08:39.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.348e-06 2021-01-15T19:08:39.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237782+0000", 2021-01-15T19:08:39.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3377000000000002e-05 2021-01-15T19:08:39.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237816+0000", 2021-01-15T19:08:39.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1600000000000003e-06 2021-01-15T19:08:39.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:39.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.237822+0000", 2021-01-15T19:08:39.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001552276 2021-01-15T19:08:39.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239374+0000", 2021-01-15T19:08:39.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:39.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:39.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.239415+0000", 2021-01-15T19:08:39.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010934268 2021-01-15T19:08:39.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250349+0000", 2021-01-15T19:08:39.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011671 2021-01-15T19:08:39.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:39.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250466+0000", 2021-01-15T19:08:39.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050409820000000003 2021-01-15T19:08:39.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255507+0000", 2021-01-15T19:08:39.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0531000000000005e-05 2021-01-15T19:08:39.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255587+0000", 2021-01-15T19:08:39.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122043 2021-01-15T19:08:39.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255709+0000", 2021-01-15T19:08:39.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017981905999999999 2021-01-15T19:08:39.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.705 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.705 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:156 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~442368] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:39.706 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:39.706 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.424500133999999, 2021-01-15T19:08:39.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759, 2021-01-15T19:08:39.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 156 2021-01-15T19:08:39.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.708 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:39.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241568+0000", 2021-01-15T19:08:39.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030551200000000003 2021-01-15T19:08:39.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241873+0000", 2021-01-15T19:08:39.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6080000000000002e-06 2021-01-15T19:08:39.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241883+0000", 2021-01-15T19:08:39.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.465e-05 2021-01-15T19:08:39.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241898+0000", 2021-01-15T19:08:39.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4010000000000002e-06 2021-01-15T19:08:39.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241904+0000", 2021-01-15T19:08:39.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6971999999999997e-05 2021-01-15T19:08:39.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.241951+0000", 2021-01-15T19:08:39.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000104244 2021-01-15T19:08:39.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242055+0000", 2021-01-15T19:08:39.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070015099999999999 2021-01-15T19:08:39.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.242755+0000", 2021-01-15T19:08:39.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0080604460000000006 2021-01-15T19:08:39.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.250816+0000", 2021-01-15T19:08:39.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00338733 2021-01-15T19:08:39.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254203+0000", 2021-01-15T19:08:39.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3444999999999998e-05 2021-01-15T19:08:39.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254287+0000", 2021-01-15T19:08:39.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012718759 2021-01-15T19:08:39.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.721 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:157 3.cs0 3:3ef45b66:::smithi00140323-11:head [append 0~401408] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:39.721 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:39.721 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.421693907000002, 2021-01-15T19:08:39.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335, 2021-01-15T19:08:39.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 157 2021-01-15T19:08:39.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.723 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:39.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244374+0000", 2021-01-15T19:08:39.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000301899 2021-01-15T19:08:39.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244676+0000", 2021-01-15T19:08:39.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6430000000000001e-06 2021-01-15T19:08:39.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244685+0000", 2021-01-15T19:08:39.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5293000000000001e-05 2021-01-15T19:08:39.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244700+0000", 2021-01-15T19:08:39.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6499999999999996e-06 2021-01-15T19:08:39.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244708+0000", 2021-01-15T19:08:39.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1064999999999997e-05 2021-01-15T19:08:39.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244789+0000", 2021-01-15T19:08:39.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117805 2021-01-15T19:08:39.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244906+0000", 2021-01-15T19:08:39.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070339100000000004 2021-01-15T19:08:39.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.245610+0000", 2021-01-15T19:08:39.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087607599999999994 2021-01-15T19:08:39.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254371+0000", 2021-01-15T19:08:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000107695 2021-01-15T19:08:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254478+0000", 2021-01-15T19:08:39.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109134 2021-01-15T19:08:39.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.254587+0000", 2021-01-15T19:08:39.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010213335 2021-01-15T19:08:39.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.736 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:158 3.cs0 3:3ef45b66:::smithi00140323-11:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected e26)", 2021-01-15T19:08:39.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:39.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.421307890000001, 2021-01-15T19:08:39.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441, 2021-01-15T19:08:39.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 158 2021-01-15T19:08:39.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.738 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:39.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244760+0000", 2021-01-15T19:08:39.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5699999999999999e-05 2021-01-15T19:08:39.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244816+0000", 2021-01-15T19:08:39.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6839999999999996e-06 2021-01-15T19:08:39.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244823+0000", 2021-01-15T19:08:39.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4626e-05 2021-01-15T19:08:39.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244838+0000", 2021-01-15T19:08:39.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6500000000000001e-06 2021-01-15T19:08:39.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244844+0000", 2021-01-15T19:08:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.004205149 2021-01-15T19:08:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249049+0000", 2021-01-15T19:08:39.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015840599999999999 2021-01-15T19:08:39.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249207+0000", 2021-01-15T19:08:39.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071027700000000004 2021-01-15T19:08:39.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.249918+0000", 2021-01-15T19:08:39.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051411900000000003 2021-01-15T19:08:39.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255059+0000", 2021-01-15T19:08:39.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045701699999999999 2021-01-15T19:08:39.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255516+0000", 2021-01-15T19:08:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8710999999999997e-05 2021-01-15T19:08:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.255614+0000", 2021-01-15T19:08:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01085441 2021-01-15T19:08:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:159 3.cs0 3:3ef45b66:::smithi00140323-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e26)", 2021-01-15T19:08:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:39.751 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.421175438999999, 2021-01-15T19:08:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195, 2021-01-15T19:08:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 159 2021-01-15T19:08:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244893+0000", 2021-01-15T19:08:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0961000000000003e-05 2021-01-15T19:08:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244933+0000", 2021-01-15T19:08:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0480000000000001e-06 2021-01-15T19:08:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244941+0000", 2021-01-15T19:08:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3711e-05 2021-01-15T19:08:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244954+0000", 2021-01-15T19:08:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2390000000000002e-06 2021-01-15T19:08:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.244959+0000", 2021-01-15T19:08:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082968859999999998 2021-01-15T19:08:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253256+0000", 2021-01-15T19:08:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000147608 2021-01-15T19:08:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.253404+0000", 2021-01-15T19:08:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0047189329999999998 2021-01-15T19:08:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258123+0000", 2021-01-15T19:08:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010107000000000001 2021-01-15T19:08:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.258224+0000", 2021-01-15T19:08:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019277389999999999 2021-01-15T19:08:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:14.260152+0000", 2021-01-15T19:08:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015259195 2021-01-15T19:08:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.764 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:201 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:39.765 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.417822112, 2021-01-15T19:08:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142, 2021-01-15T19:08:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 201 2021-01-15T19:08:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248246+0000", 2021-01-15T19:08:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1060000000000002e-05 2021-01-15T19:08:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248307+0000", 2021-01-15T19:08:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6740000000000003e-06 2021-01-15T19:08:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248316+0000", 2021-01-15T19:08:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8746e-05 2021-01-15T19:08:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248334+0000", 2021-01-15T19:08:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2294e-05 2021-01-15T19:08:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248347+0000", 2021-01-15T19:08:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6400999999999998e-05 2021-01-15T19:08:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248393+0000", 2021-01-15T19:08:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.686e-06 2021-01-15T19:08:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248400+0000", 2021-01-15T19:08:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002761197 2021-01-15T19:08:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251161+0000", 2021-01-15T19:08:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011450099999999999 2021-01-15T19:08:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.251275+0000", 2021-01-15T19:08:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062315829999999997 2021-01-15T19:08:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.257507+0000", 2021-01-15T19:08:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009261142 2021-01-15T19:08:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.778 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:202 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:39.779 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.417668737, 2021-01-15T19:08:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285, 2021-01-15T19:08:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 202 2021-01-15T19:08:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248399+0000", 2021-01-15T19:08:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5448999999999997e-05 2021-01-15T19:08:39.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248455+0000", 2021-01-15T19:08:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8290000000000002e-06 2021-01-15T19:08:39.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248461+0000", 2021-01-15T19:08:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4307e-05 2021-01-15T19:08:39.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248476+0000", 2021-01-15T19:08:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5819999999999998e-06 2021-01-15T19:08:39.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248482+0000", 2021-01-15T19:08:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1987e-05 2021-01-15T19:08:39.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248564+0000", 2021-01-15T19:08:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5850000000000002e-06 2021-01-15T19:08:39.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248572+0000", 2021-01-15T19:08:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045194099999999997 2021-01-15T19:08:39.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253091+0000", 2021-01-15T19:08:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020063399999999999 2021-01-15T19:08:39.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253292+0000", 2021-01-15T19:08:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079735020000000004 2021-01-15T19:08:39.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261266+0000", 2021-01-15T19:08:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012866285 2021-01-15T19:08:39.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.793 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:203 3.cs0 3:30718dd9:::smithi00140323-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected e28)", 2021-01-15T19:08:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.417489718999999, 2021-01-15T19:08:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438, 2021-01-15T19:08:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 203 2021-01-15T19:08:39.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:39.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248578+0000", 2021-01-15T19:08:39.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8518999999999999e-05 2021-01-15T19:08:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248607+0000", 2021-01-15T19:08:39.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8810000000000004e-06 2021-01-15T19:08:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248612+0000", 2021-01-15T19:08:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5719999999999997e-06 2021-01-15T19:08:39.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248619+0000", 2021-01-15T19:08:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5899999999999999e-06 2021-01-15T19:08:39.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248623+0000", 2021-01-15T19:08:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5608000000000001e-05 2021-01-15T19:08:39.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248668+0000", 2021-01-15T19:08:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0166000000000001e-05 2021-01-15T19:08:39.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248679+0000", 2021-01-15T19:08:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048830590000000004 2021-01-15T19:08:39.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253562+0000", 2021-01-15T19:08:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7e-05 2021-01-15T19:08:39.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253659+0000", 2021-01-15T19:08:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081539850000000007 2021-01-15T19:08:39.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.261813+0000", 2021-01-15T19:08:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01323438 2021-01-15T19:08:39.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.807 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:204 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat,delete] snapc 3=[3] ondisk+write+known_if_redirected e28)", 2021-01-15T19:08:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.417405511999998, 2021-01-15T19:08:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532, 2021-01-15T19:08:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 204 2021-01-15T19:08:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248662+0000", 2021-01-15T19:08:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8349000000000002e-05 2021-01-15T19:08:39.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248701+0000", 2021-01-15T19:08:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0620000000000001e-06 2021-01-15T19:08:39.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248709+0000", 2021-01-15T19:08:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7700000000000001e-06 2021-01-15T19:08:39.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248717+0000", 2021-01-15T19:08:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5920000000000001e-06 2021-01-15T19:08:39.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248720+0000", 2021-01-15T19:08:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0898e-05 2021-01-15T19:08:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248761+0000", 2021-01-15T19:08:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2930000000000006e-06 2021-01-15T19:08:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.248770+0000", 2021-01-15T19:08:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052185720000000003 2021-01-15T19:08:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.253989+0000", 2021-01-15T19:08:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105809 2021-01-15T19:08:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254095+0000", 2021-01-15T19:08:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078837199999999997 2021-01-15T19:08:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.254883+0000", 2021-01-15T19:08:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083722769999999992 2021-01-15T19:08:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263255+0000", 2021-01-15T19:08:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111839 2021-01-15T19:08:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263367+0000", 2021-01-15T19:08:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015969899999999999 2021-01-15T19:08:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:16.263527+0000", 2021-01-15T19:08:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014864532 2021-01-15T19:08:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.823 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:246 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:39.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.3653567, 2021-01-15T19:08:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005, 2021-01-15T19:08:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 246 2021-01-15T19:08:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300711+0000", 2021-01-15T19:08:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1804000000000001e-05 2021-01-15T19:08:39.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300763+0000", 2021-01-15T19:08:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5610000000000004e-06 2021-01-15T19:08:39.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300770+0000", 2021-01-15T19:08:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:39.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300782+0000", 2021-01-15T19:08:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4460000000000003e-06 2021-01-15T19:08:39.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300786+0000", 2021-01-15T19:08:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003399596 2021-01-15T19:08:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304186+0000", 2021-01-15T19:08:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125399 2021-01-15T19:08:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304311+0000", 2021-01-15T19:08:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057843119999999998 2021-01-15T19:08:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310095+0000", 2021-01-15T19:08:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093840550000000005 2021-01-15T19:08:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.835 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:247 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:39.836 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.365239587, 2021-01-15T19:08:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281, 2021-01-15T19:08:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 247 2021-01-15T19:08:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.838 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300828+0000", 2021-01-15T19:08:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2098999999999997e-05 2021-01-15T19:08:39.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300880+0000", 2021-01-15T19:08:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4110000000000001e-06 2021-01-15T19:08:39.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300887+0000", 2021-01-15T19:08:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.152e-05 2021-01-15T19:08:39.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300898+0000", 2021-01-15T19:08:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.904e-06 2021-01-15T19:08:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300903+0000", 2021-01-15T19:08:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039909860000000002 2021-01-15T19:08:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.304894+0000", 2021-01-15T19:08:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012664060000000001 2021-01-15T19:08:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306161+0000", 2021-01-15T19:08:39.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044999549999999999 2021-01-15T19:08:39.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310661+0000", 2021-01-15T19:08:39.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.009832281 2021-01-15T19:08:39.847 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.847 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:248 3.fs0 3:f9b5bb77:::smithi00140323-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected e36)", 2021-01-15T19:08:39.848 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.365125059, 2021-01-15T19:08:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004, 2021-01-15T19:08:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 248 2021-01-15T19:08:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300943+0000", 2021-01-15T19:08:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9626999999999998e-05 2021-01-15T19:08:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300993+0000", 2021-01-15T19:08:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1990000000000003e-06 2021-01-15T19:08:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.300999+0000", 2021-01-15T19:08:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1737e-05 2021-01-15T19:08:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301010+0000", 2021-01-15T19:08:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.006e-05 2021-01-15T19:08:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.301021+0000", 2021-01-15T19:08:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054558510000000003 2021-01-15T19:08:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306476+0000", 2021-01-15T19:08:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000193029 2021-01-15T19:08:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.306669+0000", 2021-01-15T19:08:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0046625010000000003 2021-01-15T19:08:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.311332+0000", 2021-01-15T19:08:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010389004 2021-01-15T19:08:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:252 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~647168] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:39.860 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.357386629000001, 2021-01-15T19:08:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996, 2021-01-15T19:08:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 252 2021-01-15T19:08:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.308681+0000", 2021-01-15T19:08:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041048799999999998 2021-01-15T19:08:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309092+0000", 2021-01-15T19:08:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:08:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309101+0000", 2021-01-15T19:08:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4919000000000001e-05 2021-01-15T19:08:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309116+0000", 2021-01-15T19:08:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3110000000000002e-06 2021-01-15T19:08:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309122+0000", 2021-01-15T19:08:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0148999999999997e-05 2021-01-15T19:08:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309172+0000", 2021-01-15T19:08:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013501799999999999 2021-01-15T19:08:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.309307+0000", 2021-01-15T19:08:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0015785269999999999 2021-01-15T19:08:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.310886+0000", 2021-01-15T19:08:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070233450000000003 2021-01-15T19:08:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.317909+0000", 2021-01-15T19:08:39.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2075000000000002e-05 2021-01-15T19:08:39.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318001+0000", 2021-01-15T19:08:39.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0710999999999994e-05 2021-01-15T19:08:39.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:24.318072+0000", 2021-01-15T19:08:39.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093904339999999996 2021-01-15T19:08:39.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.874 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:285 3.cs0 3:30718dd9:::smithi00140323-34:head [append 0~696320] snapc 10=[10] ondisk+write+known_if_redirected e40)", 2021-01-15T19:08:39.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:39.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.479017984, 2021-01-15T19:08:39.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093549089999999998, 2021-01-15T19:08:39.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 285 2021-01-15T19:08:39.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:39.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187050+0000", 2021-01-15T19:08:39.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041889100000000001 2021-01-15T19:08:39.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187469+0000", 2021-01-15T19:08:39.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2860999999999999e-05 2021-01-15T19:08:39.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187482+0000", 2021-01-15T19:08:39.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1379e-05 2021-01-15T19:08:39.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187503+0000", 2021-01-15T19:08:39.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0691e-05 2021-01-15T19:08:39.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187514+0000", 2021-01-15T19:08:39.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9081e-05 2021-01-15T19:08:39.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187563+0000", 2021-01-15T19:08:39.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.8070000000000002e-06 2021-01-15T19:08:39.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:39.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.187570+0000", 2021-01-15T19:08:39.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0019352589999999999 2021-01-15T19:08:39.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.189505+0000", 2021-01-15T19:08:39.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011456299999999999 2021-01-15T19:08:39.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.189620+0000", 2021-01-15T19:08:39.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000972923 2021-01-15T19:08:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.190592+0000", 2021-01-15T19:08:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056078150000000004 2021-01-15T19:08:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196200+0000", 2021-01-15T19:08:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102021 2021-01-15T19:08:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196302+0000", 2021-01-15T19:08:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000102618 2021-01-15T19:08:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:28.196405+0000", 2021-01-15T19:08:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093549089999999998 2021-01-15T19:08:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:434 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~663552] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:08:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.40751969799999999, 2021-01-15T19:08:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022929644999999999, 2021-01-15T19:08:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 434 2021-01-15T19:08:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:08:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:08:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044819800000000002 2021-01-15T19:08:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258996+0000", 2021-01-15T19:08:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4090000000000001e-05 2021-01-15T19:08:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259011+0000", 2021-01-15T19:08:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7579000000000001e-05 2021-01-15T19:08:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259038+0000", 2021-01-15T19:08:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1133e-05 2021-01-15T19:08:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259049+0000", 2021-01-15T19:08:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3312000000000006e-05 2021-01-15T19:08:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259123+0000", 2021-01-15T19:08:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3225e-05 2021-01-15T19:08:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:08:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259136+0000", 2021-01-15T19:08:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002543362 2021-01-15T19:08:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261679+0000", 2021-01-15T19:08:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012904299999999999 2021-01-15T19:08:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261808+0000", 2021-01-15T19:08:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001322642 2021-01-15T19:08:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.263131+0000", 2021-01-15T19:08:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012144998000000001 2021-01-15T19:08:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275276+0000", 2021-01-15T19:08:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060959580000000003 2021-01-15T19:08:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.281372+0000", 2021-01-15T19:08:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106105 2021-01-15T19:08:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.281478+0000", 2021-01-15T19:08:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022929644999999999 2021-01-15T19:08:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:435 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~491520] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.264066+0000", 2021-01-15T19:08:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.40200181899999998, 2021-01-15T19:08:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0187931, 2021-01-15T19:08:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 435 2021-01-15T19:08:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264066+0000", 2021-01-15T19:08:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264066+0000", 2021-01-15T19:08:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039120099999999998 2021-01-15T19:08:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264457+0000", 2021-01-15T19:08:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.022e-05 2021-01-15T19:08:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264468+0000", 2021-01-15T19:08:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4404e-05 2021-01-15T19:08:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264482+0000", 2021-01-15T19:08:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3250000000000002e-06 2021-01-15T19:08:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264489+0000", 2021-01-15T19:08:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003639691 2021-01-15T19:08:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.268129+0000", 2021-01-15T19:08:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012909900000000001 2021-01-15T19:08:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.268258+0000", 2021-01-15T19:08:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00086400500000000005 2021-01-15T19:08:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.269122+0000", 2021-01-15T19:08:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061569850000000002 2021-01-15T19:08:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275279+0000", 2021-01-15T19:08:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007489112 2021-01-15T19:08:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.282768+0000", 2021-01-15T19:08:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1057999999999995e-05 2021-01-15T19:08:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.282859+0000", 2021-01-15T19:08:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0187931 2021-01-15T19:08:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.925 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:436 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~311296] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:39.925 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.264543+0000", 2021-01-15T19:08:39.925 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.401525201, 2021-01-15T19:08:39.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018544112000000001, 2021-01-15T19:08:39.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 436 2021-01-15T19:08:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264543+0000", 2021-01-15T19:08:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264543+0000", 2021-01-15T19:08:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023832399999999999 2021-01-15T19:08:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264781+0000", 2021-01-15T19:08:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0490000000000004e-06 2021-01-15T19:08:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264789+0000", 2021-01-15T19:08:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3976e-05 2021-01-15T19:08:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264803+0000", 2021-01-15T19:08:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6219999999999997e-06 2021-01-15T19:08:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264809+0000", 2021-01-15T19:08:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059084619999999997 2021-01-15T19:08:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.270717+0000", 2021-01-15T19:08:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012303499999999999 2021-01-15T19:08:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.270840+0000", 2021-01-15T19:08:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073735300000000005 2021-01-15T19:08:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.271578+0000", 2021-01-15T19:08:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038999199999999999 2021-01-15T19:08:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275477+0000", 2021-01-15T19:08:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007502931 2021-01-15T19:08:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.282980+0000", 2021-01-15T19:08:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010644 2021-01-15T19:08:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.283087+0000", 2021-01-15T19:08:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018544112000000001 2021-01-15T19:08:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:437 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.264851+0000", 2021-01-15T19:08:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.40121715299999999, 2021-01-15T19:08:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018493532, 2021-01-15T19:08:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 437 2021-01-15T19:08:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264851+0000", 2021-01-15T19:08:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264851+0000", 2021-01-15T19:08:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1817000000000002e-05 2021-01-15T19:08:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264903+0000", 2021-01-15T19:08:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6362999999999999e-05 2021-01-15T19:08:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264919+0000", 2021-01-15T19:08:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6103000000000002e-05 2021-01-15T19:08:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264935+0000", 2021-01-15T19:08:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0829000000000001e-05 2021-01-15T19:08:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264946+0000", 2021-01-15T19:08:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083025970000000001 2021-01-15T19:08:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.273249+0000", 2021-01-15T19:08:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012209100000000001 2021-01-15T19:08:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.273371+0000", 2021-01-15T19:08:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00050558099999999996 2021-01-15T19:08:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:08:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.273876+0000", 2021-01-15T19:08:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001889562 2021-01-15T19:08:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:08:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275766+0000", 2021-01-15T19:08:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074723400000000001 2021-01-15T19:08:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:08:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.283238+0000", 2021-01-15T19:08:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010624900000000001 2021-01-15T19:08:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.283344+0000", 2021-01-15T19:08:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018493532 2021-01-15T19:08:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:438 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:08:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.264992+0000", 2021-01-15T19:08:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.40107609799999999, 2021-01-15T19:08:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021113573, 2021-01-15T19:08:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:08:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:08:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:08:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 438 2021-01-15T19:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264992+0000", 2021-01-15T19:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.264992+0000", 2021-01-15T19:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5705999999999999e-05 2021-01-15T19:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265038+0000", 2021-01-15T19:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3849999999999996e-06 2021-01-15T19:08:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:08:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265045+0000", 2021-01-15T19:08:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3059e-05 2021-01-15T19:08:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:08:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265058+0000", 2021-01-15T19:08:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3800000000000002e-06 2021-01-15T19:08:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:08:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265063+0000", 2021-01-15T19:08:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010252992000000001 2021-01-15T19:08:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275316+0000", 2021-01-15T19:08:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011428 2021-01-15T19:08:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275431+0000", 2021-01-15T19:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083974610000000002 2021-01-15T19:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.283828+0000", 2021-01-15T19:08:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122557 2021-01-15T19:08:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:08:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.283951+0000", 2021-01-15T19:08:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0021547530000000001 2021-01-15T19:08:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:08:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:08:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:08:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.286105+0000", 2021-01-15T19:08:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021113573 2021-01-15T19:08:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:08:39.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:08:39.973 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:39.974 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:39.975 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:39.973+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:39.976 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:40.075 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:40.074+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:40.154 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:40.154 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:40.155 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:40.155 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:40.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:40.168 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:40.175 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:40.174+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:40.189 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:40.210 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:40.217 INFO:tasks.rados.rados.0.smithi001.stdout:164: done (6 left) 2021-01-15T19:08:40.218 INFO:tasks.rados.rados.0.smithi001.stdout:165: done (5 left) 2021-01-15T19:08:40.218 INFO:tasks.rados.rados.0.smithi001.stdout:166: done (4 left) 2021-01-15T19:08:40.218 INFO:tasks.rados.rados.0.smithi001.stdout:167: done (3 left) 2021-01-15T19:08:40.218 INFO:tasks.rados.rados.0.smithi001.stdout:168: done (2 left) 2021-01-15T19:08:40.219 INFO:tasks.rados.rados.0.smithi001.stdout:169: done (1 left) 2021-01-15T19:08:40.219 INFO:tasks.rados.rados.0.smithi001.stdout:170: done (0 left) 2021-01-15T19:08:40.219 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 35 current snap is 16 2021-01-15T19:08:40.219 INFO:tasks.rados.rados.0.smithi001.stdout:171: seq_num 88 ranges {73728=679936,753664=778240,1531904=24576} 2021-01-15T19:08:40.226 INFO:tasks.rados.rados.0.smithi001.stdout:171: writing smithi00140323-35 from 73728 to 753664 tid 1 2021-01-15T19:08:40.233 INFO:tasks.rados.rados.0.smithi001.stdout:171: writing smithi00140323-35 from 753664 to 1531904 tid 2 2021-01-15T19:08:40.234 INFO:tasks.rados.rados.0.smithi001.stdout:171: writing smithi00140323-35 from 1531904 to 1556480 tid 3 2021-01-15T19:08:40.235 INFO:tasks.rados.rados.0.smithi001.stdout:172: copy_from oid 13 from oid 42 current snap is 16 2021-01-15T19:08:40.235 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 45 current snap is 16 2021-01-15T19:08:40.236 INFO:tasks.rados.rados.0.smithi001.stdout:173: seq_num 89 ranges {745472=696320,1441792=655360,2097152=147456} 2021-01-15T19:08:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:08:40.244 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:40.247 INFO:tasks.rados.rados.0.smithi001.stdout:173: writing smithi00140323-45 from 745472 to 1441792 tid 1 2021-01-15T19:08:40.254 INFO:tasks.rados.rados.0.smithi001.stdout:173: writing smithi00140323-45 from 1441792 to 2097152 tid 2 2021-01-15T19:08:40.256 INFO:tasks.rados.rados.0.smithi001.stdout:173: writing smithi00140323-45 from 2097152 to 2244608 tid 3 2021-01-15T19:08:40.256 INFO:tasks.rados.rados.0.smithi001.stdout:174: delete oid 37 current snap is 16 2021-01-15T19:08:40.256 INFO:tasks.rados.rados.0.smithi001.stdout:171: finishing write tid 1 to smithi00140323-35 2021-01-15T19:08:40.256 INFO:tasks.rados.rados.0.smithi001.stdout:171: finishing write tid 2 to smithi00140323-35 2021-01-15T19:08:40.256 INFO:tasks.rados.rados.0.smithi001.stdout:171: finishing write tid 3 to smithi00140323-35 2021-01-15T19:08:40.257 INFO:tasks.rados.rados.0.smithi001.stdout:171: finishing write tid 4 to smithi00140323-35 2021-01-15T19:08:40.257 INFO:tasks.rados.rados.0.smithi001.stdout:171: finishing write tid 5 to smithi00140323-35 2021-01-15T19:08:40.257 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 39 (ObjNum 88 snap 16 seq_num 88) dirty exists 2021-01-15T19:08:40.257 INFO:tasks.rados.rados.0.smithi001.stdout:171: left oid 35 (ObjNum 88 snap 16 seq_num 88) 2021-01-15T19:08:40.258 INFO:tasks.rados.rados.0.smithi001.stdout:173: finishing write tid 1 to smithi00140323-45 2021-01-15T19:08:40.261 INFO:tasks.rados.rados.0.smithi001.stdout:173: finishing write tid 2 to smithi00140323-45 2021-01-15T19:08:40.262 INFO:tasks.rados.rados.0.smithi001.stdout:173: finishing write tid 3 to smithi00140323-45 2021-01-15T19:08:40.262 INFO:tasks.rados.rados.0.smithi001.stdout:173: finishing write tid 4 to smithi00140323-45 2021-01-15T19:08:40.263 INFO:tasks.rados.rados.0.smithi001.stdout:173: finishing write tid 5 to smithi00140323-45 2021-01-15T19:08:40.263 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 48 (ObjNum 89 snap 16 seq_num 89) dirty exists 2021-01-15T19:08:40.263 INFO:tasks.rados.rados.0.smithi001.stdout:173: left oid 45 (ObjNum 89 snap 16 seq_num 89) 2021-01-15T19:08:40.273 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:40.276 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:40.275+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:40.294 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:40.316 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:40.318 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:40.318 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:40.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:40.336 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.338 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.339 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:40.377 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:40.376+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:40.468 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:40.469 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:40.477 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:40.476+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:40.478 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:40.578 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:40.577+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:40.610 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:40.611 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:40.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:40.679 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:40.678+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:40.753 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:40.762 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:40.780 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:40.779+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:40.883 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:40.880+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:40.935 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:40.935 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:40.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:40.987 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:40.985+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:41.083 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:41.081+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:41.118 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:41.118 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:41.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:41.215 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:41.213+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:171: done (3 left) 2021-01-15T19:08:41.249 INFO:tasks.rados.rados.0.smithi001.stdout:173: done (2 left) 2021-01-15T19:08:41.249 INFO:tasks.rados.rados.0.smithi001.stdout:174: done (1 left) 2021-01-15T19:08:41.249 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 21 current snap is 16 2021-01-15T19:08:41.250 INFO:tasks.rados.rados.0.smithi001.stdout:175: seq_num 91 ranges {344064=729088,1073152=425984,1499136=376832} 2021-01-15T19:08:41.254 INFO:tasks.rados.rados.0.smithi001.stdout:175: writing smithi00140323-21 from 344064 to 1073152 tid 1 2021-01-15T19:08:41.256 INFO:tasks.rados.rados.0.smithi001.stdout:175: writing smithi00140323-21 from 1073152 to 1499136 tid 2 2021-01-15T19:08:41.259 INFO:tasks.rados.rados.0.smithi001.stdout:175: writing smithi00140323-21 from 1499136 to 1875968 tid 3 2021-01-15T19:08:41.259 INFO:tasks.rados.rados.0.smithi001.stdout:176: rmattr oid 11 current snap is 16 2021-01-15T19:08:41.259 INFO:tasks.rados.rados.0.smithi001.stdout:172: finishing copy_from to smithi00140323-13 2021-01-15T19:08:41.260 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:08:41.260 INFO:tasks.rados.rados.0.smithi001.stdout:172: finishing copy_from racing read to smithi00140323-13 2021-01-15T19:08:41.260 INFO:tasks.rados.rados.0.smithi001.stdout:172: done (2 left) 2021-01-15T19:08:41.260 INFO:tasks.rados.rados.0.smithi001.stdout:177: read oid 26 snap 7 2021-01-15T19:08:41.261 INFO:tasks.rados.rados.0.smithi001.stdout:177: expect (ObjNum 25 snap 0 seq_num 25) 2021-01-15T19:08:41.261 INFO:tasks.rados.rados.0.smithi001.stdout:178: delete oid 6 current snap is 16 2021-01-15T19:08:41.264 INFO:tasks.rados.rados.0.smithi001.stdout:178: done (3 left) 2021-01-15T19:08:41.265 INFO:tasks.rados.rados.0.smithi001.stdout:179: read oid 5 snap -1 2021-01-15T19:08:41.265 INFO:tasks.rados.rados.0.smithi001.stdout:179: expect (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:08:41.265 INFO:tasks.rados.rados.0.smithi001.stdout:180: read oid 48 snap -1 2021-01-15T19:08:41.265 INFO:tasks.rados.rados.0.smithi001.stdout:180: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:08:41.266 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 37 current snap is 16 2021-01-15T19:08:41.266 INFO:tasks.rados.rados.0.smithi001.stdout:181: seq_num 93 ranges {0=761856,761856=647168,1409024=155648} 2021-01-15T19:08:41.269 INFO:tasks.rados.rados.0.smithi001.stdout:181: writing smithi00140323-37 from 0 to 761856 tid 1 2021-01-15T19:08:41.274 INFO:tasks.rados.rados.0.smithi001.stdout:181: writing smithi00140323-37 from 761856 to 1409024 tid 2 2021-01-15T19:08:41.275 INFO:tasks.rados.rados.0.smithi001.stdout:181: writing smithi00140323-37 from 1409024 to 1564672 tid 3 2021-01-15T19:08:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 50 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:08:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:177: expect (ObjNum 25 snap 0 seq_num 25) 2021-01-15T19:08:41.285 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:41.283+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:41.292 INFO:tasks.rados.rados.0.smithi001.stdout:175: finishing write tid 1 to smithi00140323-21 2021-01-15T19:08:41.292 INFO:tasks.rados.rados.0.smithi001.stdout:179: expect (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:08:41.303 INFO:tasks.rados.rados.0.smithi001.stdout:175: finishing write tid 2 to smithi00140323-21 2021-01-15T19:08:41.303 INFO:tasks.rados.rados.0.smithi001.stdout:175: finishing write tid 3 to smithi00140323-21 2021-01-15T19:08:41.303 INFO:tasks.rados.rados.0.smithi001.stdout:175: finishing write tid 4 to smithi00140323-21 2021-01-15T19:08:41.304 INFO:tasks.rados.rados.0.smithi001.stdout:175: finishing write tid 5 to smithi00140323-21 2021-01-15T19:08:41.304 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 25 (ObjNum 91 snap 16 seq_num 91) dirty exists 2021-01-15T19:08:41.304 INFO:tasks.rados.rados.0.smithi001.stdout:175: left oid 21 (ObjNum 91 snap 16 seq_num 91) 2021-01-15T19:08:41.304 INFO:tasks.rados.rados.0.smithi001.stdout:175: done (5 left) 2021-01-15T19:08:41.305 INFO:tasks.rados.rados.0.smithi001.stdout:176: done (4 left) 2021-01-15T19:08:41.305 INFO:tasks.rados.rados.0.smithi001.stdout:177: done (3 left) 2021-01-15T19:08:41.305 INFO:tasks.rados.rados.0.smithi001.stdout:179: done (2 left) 2021-01-15T19:08:41.305 INFO:tasks.rados.rados.0.smithi001.stdout:182: read oid 47 snap -1 2021-01-15T19:08:41.306 INFO:tasks.rados.rados.0.smithi001.stdout:182: expect (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:08:41.306 INFO:tasks.rados.rados.0.smithi001.stdout:183: delete oid 7 current snap is 16 2021-01-15T19:08:41.306 INFO:tasks.rados.rados.0.smithi001.stdout:181: finishing write tid 1 to smithi00140323-37 2021-01-15T19:08:41.306 INFO:tasks.rados.rados.0.smithi001.stdout:181: finishing write tid 2 to smithi00140323-37 2021-01-15T19:08:41.307 INFO:tasks.rados.rados.0.smithi001.stdout:181: finishing write tid 3 to smithi00140323-37 2021-01-15T19:08:41.307 INFO:tasks.rados.rados.0.smithi001.stdout:181: finishing write tid 4 to smithi00140323-37 2021-01-15T19:08:41.311 INFO:tasks.rados.rados.0.smithi001.stdout:181: finishing write tid 5 to smithi00140323-37 2021-01-15T19:08:41.311 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 24 (ObjNum 93 snap 16 seq_num 93) dirty exists 2021-01-15T19:08:41.311 INFO:tasks.rados.rados.0.smithi001.stdout:181: left oid 37 (ObjNum 93 snap 16 seq_num 93) 2021-01-15T19:08:41.311 INFO:tasks.rados.rados.0.smithi001.stdout:180: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:08:41.331 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:41.331 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:41.343 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:41.386 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:41.385+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:41.488 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:41.486+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:41.509 INFO:tasks.rados.rados.0.smithi001.stdout:182: expect (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:08:41.518 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:41.518 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:41.527 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:41.588 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:41.586+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:41.636 INFO:tasks.rados.rados.0.smithi001.stdout:180: done (3 left) 2021-01-15T19:08:41.636 INFO:tasks.rados.rados.0.smithi001.stdout:181: done (2 left) 2021-01-15T19:08:41.637 INFO:tasks.rados.rados.0.smithi001.stdout:182: done (1 left) 2021-01-15T19:08:41.637 INFO:tasks.rados.rados.0.smithi001.stdout:183: done (0 left) 2021-01-15T19:08:41.637 INFO:tasks.rados.rados.0.smithi001.stdout:184: copy_from oid 3 from oid 34 current snap is 16 2021-01-15T19:08:41.638 INFO:tasks.rados.rados.0.smithi001.stdout:185: snap_remove snap 15 2021-01-15T19:08:41.664 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:41.665 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:41.676 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:41.715 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:41.714+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:41.790 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:41.789+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:41.839 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:41.839 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:41.849 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:41.894 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:41.892+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:41.995 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:41.993+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:42.018 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:42.018 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:42.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:42.095 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:42.094+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:42.192 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:42.193 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:42.196 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:42.194+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:42.202 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:42.232 INFO:tasks.rados.rados.0.smithi001.stdout:185: done (1 left) 2021-01-15T19:08:42.232 INFO:tasks.rados.rados.0.smithi001.stdout:186: delete oid 5 current snap is 16 2021-01-15T19:08:42.232 INFO:tasks.rados.rados.0.smithi001.stdout:184: finishing copy_from to smithi00140323-3 2021-01-15T19:08:42.233 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:08:42.233 INFO:tasks.rados.rados.0.smithi001.stdout:184: finishing copy_from racing read to smithi00140323-3 2021-01-15T19:08:42.252 INFO:tasks.rados.rados.0.smithi001.stdout:184: done (1 left) 2021-01-15T19:08:42.253 INFO:tasks.rados.rados.0.smithi001.stdout:186: done (0 left) 2021-01-15T19:08:42.253 INFO:tasks.rados.rados.0.smithi001.stdout:187: snap_remove snap 7 2021-01-15T19:08:42.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:08:42.298 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:42.295+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:42.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:42.451 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:42.451 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:42.452 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:42.454 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:42.453+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:42.559 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:42.556+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:42.600 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:42.601 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:42.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:42.606 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:42.621 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:42.645 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:42.674 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:42.657+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:42.674 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:42.683 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:42.703 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:42.732 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:42.745 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:42.746 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.746 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.746 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.746 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.747 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.747 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.747 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.747 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:42.766 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:42.760+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:42.786 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:42.787 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:42.787 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:42.787 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:42.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:42.870 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:42.866+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:42.936 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:42.938 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:42.938 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:42.938 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:42.938 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:42.949 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:42.968 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:42.967+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:43.079 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:43.073+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:43.132 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:43.132 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:43.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:43.133 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:43.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.133 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=71, reqid=client.4418.0:183, at_version=27'8, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:43.133 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:43.133 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.877191108000002, 2021-01-15T19:08:43.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003, 2021-01-15T19:08:43.134 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.134 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:43.134 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:43.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251697+0000", 2021-01-15T19:08:43.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190392 2021-01-15T19:08:43.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251887+0000", 2021-01-15T19:08:43.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6526999999999999e-05 2021-01-15T19:08:43.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251914+0000", 2021-01-15T19:08:43.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8068999999999999e-05 2021-01-15T19:08:43.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251932+0000", 2021-01-15T19:08:43.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7269999999999996e-06 2021-01-15T19:08:43.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.251940+0000", 2021-01-15T19:08:43.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012910800000000001 2021-01-15T19:08:43.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252070+0000", 2021-01-15T19:08:43.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2417e-05 2021-01-15T19:08:43.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.252082+0000", 2021-01-15T19:08:43.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050434080000000001 2021-01-15T19:08:43.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257125+0000", 2021-01-15T19:08:43.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6032999999999998e-05 2021-01-15T19:08:43.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.257171+0000", 2021-01-15T19:08:43.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054746810000000003 2021-01-15T19:08:43.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:43.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:43.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.873339748999999, 2021-01-15T19:08:43.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:43.154 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.154 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.155 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.155 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:43.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.155 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:43.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:43.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:43.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:43.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:43.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:43.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:43.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:43.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:43.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:43.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:43.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:43.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 27/21 ECSubWrite(tid=79, reqid=client.4418.0:185, at_version=27'10, trim_to=0'0, roll_forward_to=26'6))", 2021-01-15T19:08:43.246 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:43.252 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.87214479, 2021-01-15T19:08:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999, 2021-01-15T19:08:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256743+0000", 2021-01-15T19:08:43.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5377000000000001e-05 2021-01-15T19:08:43.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256768+0000", 2021-01-15T19:08:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3387000000000001e-05 2021-01-15T19:08:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256782+0000", 2021-01-15T19:08:43.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6764999999999998e-05 2021-01-15T19:08:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256819+0000", 2021-01-15T19:08:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256826+0000", 2021-01-15T19:08:43.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9475999999999998e-05 2021-01-15T19:08:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256886+0000", 2021-01-15T19:08:43.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1929e-05 2021-01-15T19:08:43.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256898+0000", 2021-01-15T19:08:43.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055282589999999998 2021-01-15T19:08:43.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262426+0000", 2021-01-15T19:08:43.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2537999999999997e-05 2021-01-15T19:08:43.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262479+0000", 2021-01-15T19:08:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057354889999999999 2021-01-15T19:08:43.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.307 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.870613109000001, 2021-01-15T19:08:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:43.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:43.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:43.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.8696889, 2021-01-15T19:08:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.357 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.869464937, 2021-01-15T19:08:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.845105904, 2021-01-15T19:08:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.8429267, 2021-01-15T19:08:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.842172336000001, 2021-01-15T19:08:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.433 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.842052169999999, 2021-01-15T19:08:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:318 3.1s0 3:8b30e322:::smithi00140323-24:head [copy-from ver 8] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.932851573000001, 2021-01-15T19:08:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729, 2021-01-15T19:08:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 318 2021-01-15T19:08:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2021-01-15T19:08:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196102+0000", 2021-01-15T19:08:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6286e-05 2021-01-15T19:08:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196119+0000", 2021-01-15T19:08:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0645e-05 2021-01-15T19:08:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196139+0000", 2021-01-15T19:08:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576000000000001e-05 2021-01-15T19:08:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196151+0000", 2021-01-15T19:08:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017478200000000001 2021-01-15T19:08:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196326+0000", 2021-01-15T19:08:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019917200000000001 2021-01-15T19:08:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.483 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:43.484 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:43.484 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2021-01-15T19:08:43.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:43.492 INFO:tasks.rados.rados.0.smithi001.stdout:187: done (0 left) 2021-01-15T19:08:43.492 INFO:tasks.rados.rados.0.smithi001.stdout:188: read oid 15 snap 13 2021-01-15T19:08:43.492 INFO:tasks.rados.rados.0.smithi001.stdout:188: expect (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:08:43.493 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 19 current snap is 16 2021-01-15T19:08:43.493 INFO:tasks.rados.rados.0.smithi001.stdout:189: seq_num 96 ranges {2392064=409600,2801664=696320,3497984=507904} 2021-01-15T19:08:43.493 INFO:tasks.rados.rados.0.smithi001.stdout:189: writing smithi00140323-19 from 2392064 to 2801664 tid 1 2021-01-15T19:08:43.493 INFO:tasks.rados.rados.0.smithi001.stdout:189: writing smithi00140323-19 from 2801664 to 3497984 tid 2 2021-01-15T19:08:43.493 INFO:tasks.rados.rados.0.smithi001.stdout:189: writing smithi00140323-19 from 3497984 to 4005888 tid 3 2021-01-15T19:08:43.494 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 16 2021-01-15T19:08:43.494 INFO:tasks.rados.rados.0.smithi001.stdout:190: seq_num 97 ranges {1146880=434176,1581056=581632,2162688=540672,2703360=81920} 2021-01-15T19:08:43.494 INFO:tasks.rados.rados.0.smithi001.stdout:190: writing smithi00140323-14 from 1146880 to 1581056 tid 1 2021-01-15T19:08:43.494 INFO:tasks.rados.rados.0.smithi001.stdout:190: writing smithi00140323-14 from 1581056 to 2162688 tid 2 2021-01-15T19:08:43.494 INFO:tasks.rados.rados.0.smithi001.stdout:190: writing smithi00140323-14 from 2162688 to 2703360 tid 3 2021-01-15T19:08:43.495 INFO:tasks.rados.rados.0.smithi001.stdout:190: writing smithi00140323-14 from 2703360 to 2785280 tid 4 2021-01-15T19:08:43.495 INFO:tasks.rados.rados.0.smithi001.stdout:189: finishing write tid 1 to smithi00140323-19 2021-01-15T19:08:43.495 INFO:tasks.rados.rados.0.smithi001.stdout:189: finishing write tid 2 to smithi00140323-19 2021-01-15T19:08:43.495 INFO:tasks.rados.rados.0.smithi001.stdout:189: finishing write tid 3 to smithi00140323-19 2021-01-15T19:08:43.495 INFO:tasks.rados.rados.0.smithi001.stdout:189: finishing write tid 4 to smithi00140323-19 2021-01-15T19:08:43.496 INFO:tasks.rados.rados.0.smithi001.stdout:189: finishing write tid 5 to smithi00140323-19 2021-01-15T19:08:43.496 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 37 (ObjNum 96 snap 16 seq_num 96) dirty exists 2021-01-15T19:08:43.496 INFO:tasks.rados.rados.0.smithi001.stdout:189: left oid 19 (ObjNum 96 snap 16 seq_num 96) 2021-01-15T19:08:43.496 INFO:tasks.rados.rados.0.smithi001.stdout:191: read oid 21 snap -1 2021-01-15T19:08:43.496 INFO:tasks.rados.rados.0.smithi001.stdout:191: expect (ObjNum 91 snap 16 seq_num 91) 2021-01-15T19:08:43.497 INFO:tasks.rados.rados.0.smithi001.stdout:189: done (3 left) 2021-01-15T19:08:43.497 INFO:tasks.rados.rados.0.smithi001.stdout:192: copy_from oid 44 from oid 3 current snap is 16 2021-01-15T19:08:43.497 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 16 2021-01-15T19:08:43.497 INFO:tasks.rados.rados.0.smithi001.stdout:193: seq_num 98 ranges {696320=450560,1146880=466944,1613824=737280} 2021-01-15T19:08:43.497 INFO:tasks.rados.rados.0.smithi001.stdout:193: writing smithi00140323-13 from 696320 to 1146880 tid 1 2021-01-15T19:08:43.498 INFO:tasks.rados.rados.0.smithi001.stdout:193: writing smithi00140323-13 from 1146880 to 1613824 tid 2 2021-01-15T19:08:43.498 INFO:tasks.rados.rados.0.smithi001.stdout:193: writing smithi00140323-13 from 1613824 to 2351104 tid 3 2021-01-15T19:08:43.498 INFO:tasks.rados.rados.0.smithi001.stdout:194: read oid 45 snap -1 2021-01-15T19:08:43.498 INFO:tasks.rados.rados.0.smithi001.stdout:194: expect (ObjNum 89 snap 16 seq_num 89) 2021-01-15T19:08:43.498 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 16 2021-01-15T19:08:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:190: finishing write tid 1 to smithi00140323-14 2021-01-15T19:08:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:195: seq_num 99 ranges {3039232=466944,3506176=802816,4308992=401408} 2021-01-15T19:08:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:195: writing smithi00140323-39 from 3039232 to 3506176 tid 1 2021-01-15T19:08:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:195: writing smithi00140323-39 from 3506176 to 4308992 tid 2 2021-01-15T19:08:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:195: writing smithi00140323-39 from 4308992 to 4710400 tid 3 2021-01-15T19:08:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 8 current snap is 16 2021-01-15T19:08:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:190: finishing write tid 2 to smithi00140323-14 2021-01-15T19:08:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:190: finishing write tid 3 to smithi00140323-14 2021-01-15T19:08:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:190: finishing write tid 4 to smithi00140323-14 2021-01-15T19:08:43.501 INFO:tasks.rados.rados.0.smithi001.stdout:190: finishing write tid 5 to smithi00140323-14 2021-01-15T19:08:43.501 INFO:tasks.rados.rados.0.smithi001.stdout:193: finishing write tid 1 to smithi00140323-13 2021-01-15T19:08:43.501 INFO:tasks.rados.rados.0.smithi001.stdout:190: finishing write tid 6 to smithi00140323-14 2021-01-15T19:08:43.501 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 43 (ObjNum 97 snap 16 seq_num 97) dirty exists 2021-01-15T19:08:43.501 INFO:tasks.rados.rados.0.smithi001.stdout:190: left oid 14 (ObjNum 97 snap 16 seq_num 97) 2021-01-15T19:08:43.502 INFO:tasks.rados.rados.0.smithi001.stdout:193: finishing write tid 2 to smithi00140323-13 2021-01-15T19:08:43.502 INFO:tasks.rados.rados.0.smithi001.stdout:191: expect (ObjNum 91 snap 16 seq_num 91) 2021-01-15T19:08:43.502 INFO:tasks.rados.rados.0.smithi001.stdout:193: finishing write tid 3 to smithi00140323-13 2021-01-15T19:08:43.502 INFO:tasks.rados.rados.0.smithi001.stdout:193: finishing write tid 4 to smithi00140323-13 2021-01-15T19:08:43.502 INFO:tasks.rados.rados.0.smithi001.stdout:193: finishing write tid 5 to smithi00140323-13 2021-01-15T19:08:43.503 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 46 (ObjNum 98 snap 16 seq_num 98) dirty exists 2021-01-15T19:08:43.503 INFO:tasks.rados.rados.0.smithi001.stdout:193: left oid 13 (ObjNum 98 snap 16 seq_num 98) 2021-01-15T19:08:43.503 INFO:tasks.rados.rados.0.smithi001.stdout:192: finishing copy_from to smithi00140323-44 2021-01-15T19:08:43.503 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 44 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:08:43.503 INFO:tasks.rados.rados.0.smithi001.stdout:192: finishing copy_from racing read to smithi00140323-44 2021-01-15T19:08:43.504 INFO:tasks.rados.rados.0.smithi001.stdout:194: expect (ObjNum 89 snap 16 seq_num 89) 2021-01-15T19:08:43.504 INFO:tasks.rados.rados.0.smithi001.stdout:195: finishing write tid 1 to smithi00140323-39 2021-01-15T19:08:43.504 INFO:tasks.rados.rados.0.smithi001.stdout:195: finishing write tid 2 to smithi00140323-39 2021-01-15T19:08:43.504 INFO:tasks.rados.rados.0.smithi001.stdout:195: finishing write tid 3 to smithi00140323-39 2021-01-15T19:08:43.504 INFO:tasks.rados.rados.0.smithi001.stdout:195: finishing write tid 4 to smithi00140323-39 2021-01-15T19:08:43.504 INFO:tasks.rados.rados.0.smithi001.stdout:195: finishing write tid 5 to smithi00140323-39 2021-01-15T19:08:43.505 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 37 (ObjNum 99 snap 16 seq_num 99) dirty exists 2021-01-15T19:08:43.505 INFO:tasks.rados.rados.0.smithi001.stdout:195: left oid 39 (ObjNum 99 snap 16 seq_num 99) 2021-01-15T19:08:43.505 INFO:tasks.rados.rados.0.smithi001.stdout:196: seq_num 100 ranges {122880=483328,606208=688128,1294336=516096} 2021-01-15T19:08:43.505 INFO:tasks.rados.rados.0.smithi001.stdout:196: writing smithi00140323-8 from 122880 to 606208 tid 1 2021-01-15T19:08:43.505 INFO:tasks.rados.rados.0.smithi001.stdout:196: writing smithi00140323-8 from 606208 to 1294336 tid 2 2021-01-15T19:08:43.506 INFO:tasks.rados.rados.0.smithi001.stdout:196: writing smithi00140323-8 from 1294336 to 1810432 tid 3 2021-01-15T19:08:43.506 INFO:tasks.rados.rados.0.smithi001.stdout:190: done (7 left) 2021-01-15T19:08:43.506 INFO:tasks.rados.rados.0.smithi001.stdout:191: done (6 left) 2021-01-15T19:08:43.506 INFO:tasks.rados.rados.0.smithi001.stdout:192: done (5 left) 2021-01-15T19:08:43.506 INFO:tasks.rados.rados.0.smithi001.stdout:193: done (4 left) 2021-01-15T19:08:43.507 INFO:tasks.rados.rados.0.smithi001.stdout:194: done (3 left) 2021-01-15T19:08:43.507 INFO:tasks.rados.rados.0.smithi001.stdout:195: done (2 left) 2021-01-15T19:08:43.507 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 5 current snap is 16 2021-01-15T19:08:43.507 INFO:tasks.rados.rados.0.smithi001.stdout:197: seq_num 101 ranges {0=499712,499712=573440,1073152=524288,1597440=106496} 2021-01-15T19:08:43.507 INFO:tasks.rados.rados.0.smithi001.stdout:197: writing smithi00140323-5 from 0 to 499712 tid 1 2021-01-15T19:08:43.508 INFO:tasks.rados.rados.0.smithi001.stdout:197: writing smithi00140323-5 from 499712 to 1073152 tid 2 2021-01-15T19:08:43.508 INFO:tasks.rados.rados.0.smithi001.stdout:197: writing smithi00140323-5 from 1073152 to 1597440 tid 3 2021-01-15T19:08:43.508 INFO:tasks.rados.rados.0.smithi001.stdout:197: writing smithi00140323-5 from 1597440 to 1703936 tid 4 2021-01-15T19:08:43.508 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 11 current snap is 16 2021-01-15T19:08:43.508 INFO:tasks.rados.rados.0.smithi001.stdout:196: finishing write tid 1 to smithi00140323-8 2021-01-15T19:08:43.509 INFO:tasks.rados.rados.0.smithi001.stdout:198: seq_num 102 ranges {319488=516096,835584=450560,1286144=729088,2015232=24576} 2021-01-15T19:08:43.509 INFO:tasks.rados.rados.0.smithi001.stdout:198: writing smithi00140323-11 from 319488 to 835584 tid 1 2021-01-15T19:08:43.509 INFO:tasks.rados.rados.0.smithi001.stdout:198: writing smithi00140323-11 from 835584 to 1286144 tid 2 2021-01-15T19:08:43.509 INFO:tasks.rados.rados.0.smithi001.stdout:198: writing smithi00140323-11 from 1286144 to 2015232 tid 3 2021-01-15T19:08:43.509 INFO:tasks.rados.rados.0.smithi001.stdout:198: writing smithi00140323-11 from 2015232 to 2039808 tid 4 2021-01-15T19:08:43.510 INFO:tasks.rados.rados.0.smithi001.stdout:199: rollback oid 40 current snap is 16 2021-01-15T19:08:43.510 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 40 to 13 2021-01-15T19:08:43.510 INFO:tasks.rados.rados.0.smithi001.stdout:196: finishing write tid 2 to smithi00140323-8 2021-01-15T19:08:43.510 INFO:tasks.rados.rados.0.smithi001.stdout:196: finishing write tid 3 to smithi00140323-8 2021-01-15T19:08:43.510 INFO:tasks.rados.rados.0.smithi001.stdout:196: finishing write tid 4 to smithi00140323-8 2021-01-15T19:08:43.511 INFO:tasks.rados.rados.0.smithi001.stdout:196: finishing write tid 5 to smithi00140323-8 2021-01-15T19:08:43.511 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 31 (ObjNum 100 snap 16 seq_num 100) dirty exists 2021-01-15T19:08:43.511 INFO:tasks.rados.rados.0.smithi001.stdout:196: left oid 8 (ObjNum 100 snap 16 seq_num 100) 2021-01-15T19:08:43.511 INFO:tasks.rados.rados.0.smithi001.stdout:197: finishing write tid 1 to smithi00140323-5 2021-01-15T19:08:43.511 INFO:tasks.rados.rados.0.smithi001.stdout:196: done (4 left) 2021-01-15T19:08:43.512 INFO:tasks.rados.rados.0.smithi001.stdout:200: read oid 25 snap -1 2021-01-15T19:08:43.512 INFO:tasks.rados.rados.0.smithi001.stdout:200: expect (ObjNum 45 snap 0 seq_num 45) 2021-01-15T19:08:43.512 INFO:tasks.rados.rados.0.smithi001.stdout:201: snap_create 2021-01-15T19:08:43.512 INFO:tasks.rados.rados.0.smithi001.stdout:197: finishing write tid 2 to smithi00140323-5 2021-01-15T19:08:43.512 INFO:tasks.rados.rados.0.smithi001.stdout:197: finishing write tid 3 to smithi00140323-5 2021-01-15T19:08:43.513 INFO:tasks.rados.rados.0.smithi001.stdout:197: finishing write tid 4 to smithi00140323-5 2021-01-15T19:08:43.513 INFO:tasks.rados.rados.0.smithi001.stdout:197: finishing write tid 5 to smithi00140323-5 2021-01-15T19:08:43.513 INFO:tasks.rados.rados.0.smithi001.stdout:197: finishing write tid 6 to smithi00140323-5 2021-01-15T19:08:43.513 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 25 (ObjNum 101 snap 16 seq_num 101) dirty exists 2021-01-15T19:08:43.513 INFO:tasks.rados.rados.0.smithi001.stdout:197: left oid 5 (ObjNum 101 snap 16 seq_num 101) 2021-01-15T19:08:43.514 INFO:tasks.rados.rados.0.smithi001.stdout:198: finishing write tid 1 to smithi00140323-11 2021-01-15T19:08:43.514 INFO:tasks.rados.rados.0.smithi001.stdout:198: finishing write tid 2 to smithi00140323-11 2021-01-15T19:08:43.514 INFO:tasks.rados.rados.0.smithi001.stdout:198: finishing write tid 3 to smithi00140323-11 2021-01-15T19:08:43.514 INFO:tasks.rados.rados.0.smithi001.stdout:200: expect (ObjNum 45 snap 0 seq_num 45) 2021-01-15T19:08:43.514 INFO:tasks.rados.rados.0.smithi001.stdout:198: finishing write tid 4 to smithi00140323-11 2021-01-15T19:08:43.515 INFO:tasks.rados.rados.0.smithi001.stdout:198: finishing write tid 5 to smithi00140323-11 2021-01-15T19:08:43.515 INFO:tasks.rados.rados.0.smithi001.stdout:198: finishing write tid 6 to smithi00140323-11 2021-01-15T19:08:43.515 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 67 (ObjNum 102 snap 16 seq_num 102) dirty exists 2021-01-15T19:08:43.515 INFO:tasks.rados.rados.0.smithi001.stdout:198: left oid 11 (ObjNum 102 snap 16 seq_num 102) 2021-01-15T19:08:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:08:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196525+0000", 2021-01-15T19:08:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099001019999999992 2021-01-15T19:08:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.206425+0000", 2021-01-15T19:08:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043897240000000002 2021-01-15T19:08:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210815+0000", 2021-01-15T19:08:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023466619999999998 2021-01-15T19:08:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213161+0000", 2021-01-15T19:08:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108811 2021-01-15T19:08:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213270+0000", 2021-01-15T19:08:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729 2021-01-15T19:08:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:319 3.1s0 3:8b30e322:::smithi00140323-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.932688655, 2021-01-15T19:08:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001, 2021-01-15T19:08:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 319 2021-01-15T19:08:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6667e-05 2021-01-15T19:08:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196236+0000", 2021-01-15T19:08:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196243+0000", 2021-01-15T19:08:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4992e-05 2021-01-15T19:08:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196258+0000", 2021-01-15T19:08:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6110000000000001e-06 2021-01-15T19:08:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196264+0000", 2021-01-15T19:08:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046740399999999999 2021-01-15T19:08:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196731+0000", 2021-01-15T19:08:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1105000000000004e-05 2021-01-15T19:08:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196812+0000", 2021-01-15T19:08:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011315451000000001 2021-01-15T19:08:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208128+0000", 2021-01-15T19:08:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010873 2021-01-15T19:08:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208237+0000", 2021-01-15T19:08:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058576419999999997 2021-01-15T19:08:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214094+0000", 2021-01-15T19:08:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119562 2021-01-15T19:08:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214214+0000", 2021-01-15T19:08:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012588000000000001 2021-01-15T19:08:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214340+0000", 2021-01-15T19:08:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001 2021-01-15T19:08:43.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.537 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.538 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:323 3.1s0 3:80f33866:::smithi00140323-14:head [delete] snapc 11=[11,10] ondisk+write+peerstat_old+known_if_redirected e41)", 2021-01-15T19:08:43.538 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:43.538 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.932580029, 2021-01-15T19:08:43.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061698250000000003, 2021-01-15T19:08:43.538 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:43.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.539 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 323 2021-01-15T19:08:43.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.540 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:43.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196308+0000", 2021-01-15T19:08:43.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6786999999999999e-05 2021-01-15T19:08:43.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196345+0000", 2021-01-15T19:08:43.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6189999999999996e-06 2021-01-15T19:08:43.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196351+0000", 2021-01-15T19:08:43.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1976e-05 2021-01-15T19:08:43.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196363+0000", 2021-01-15T19:08:43.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9249999999999995e-06 2021-01-15T19:08:43.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196371+0000", 2021-01-15T19:08:43.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047688599999999999 2021-01-15T19:08:43.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196848+0000", 2021-01-15T19:08:43.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010886 2021-01-15T19:08:43.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196957+0000", 2021-01-15T19:08:43.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087102499999999999 2021-01-15T19:08:43.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.197828+0000", 2021-01-15T19:08:43.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022233589999999998 2021-01-15T19:08:43.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.200051+0000", 2021-01-15T19:08:43.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023389800000000001 2021-01-15T19:08:43.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:43.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202390+0000", 2021-01-15T19:08:43.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7408000000000001e-05 2021-01-15T19:08:43.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.202478+0000", 2021-01-15T19:08:43.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061698250000000003 2021-01-15T19:08:43.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.553 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.553 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=129, reqid=client.4418.0:364, at_version=45'25, trim_to=0'0, roll_forward_to=39'21))", 2021-01-15T19:08:43.554 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:43.554 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9320878130000008, 2021-01-15T19:08:43.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008030611, 2021-01-15T19:08:43.554 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.555 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:43.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:43.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196800+0000", 2021-01-15T19:08:43.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015823399999999999 2021-01-15T19:08:43.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196958+0000", 2021-01-15T19:08:43.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1186999999999999e-05 2021-01-15T19:08:43.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196979+0000", 2021-01-15T19:08:43.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5041999999999999e-05 2021-01-15T19:08:43.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196994+0000", 2021-01-15T19:08:43.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7460000000000003e-06 2021-01-15T19:08:43.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197001+0000", 2021-01-15T19:08:43.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021975599999999999 2021-01-15T19:08:43.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197221+0000", 2021-01-15T19:08:43.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2945999999999999e-05 2021-01-15T19:08:43.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197234+0000", 2021-01-15T19:08:43.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075394920000000001 2021-01-15T19:08:43.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204773+0000", 2021-01-15T19:08:43.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7207999999999998e-05 2021-01-15T19:08:43.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204831+0000", 2021-01-15T19:08:43.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008030611 2021-01-15T19:08:43.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.570 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/21 ECSubWrite(tid=130, reqid=client.4418.0:365, at_version=45'26, trim_to=0'0, roll_forward_to=45'23))", 2021-01-15T19:08:43.570 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:43.570 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9300392469999998, 2021-01-15T19:08:43.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060228970000000001, 2021-01-15T19:08:43.570 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.571 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:43.571 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:43.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198849+0000", 2021-01-15T19:08:43.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7685e-05 2021-01-15T19:08:43.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198876+0000", 2021-01-15T19:08:43.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2933e-05 2021-01-15T19:08:43.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198889+0000", 2021-01-15T19:08:43.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5310000000000007e-06 2021-01-15T19:08:43.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198899+0000", 2021-01-15T19:08:43.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:08:43.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198904+0000", 2021-01-15T19:08:43.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7002999999999997e-05 2021-01-15T19:08:43.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198951+0000", 2021-01-15T19:08:43.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1732e-05 2021-01-15T19:08:43.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.198963+0000", 2021-01-15T19:08:43.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058719660000000002 2021-01-15T19:08:43.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204835+0000", 2021-01-15T19:08:43.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6568e-05 2021-01-15T19:08:43.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.204871+0000", 2021-01-15T19:08:43.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0060228970000000001 2021-01-15T19:08:43.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.581 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.582 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:442 3.4s0 3:2ea4cd8e:::smithi00140323-4:head [read 0~57344,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:43.582 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264015+0000", 2021-01-15T19:08:43.582 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8648732579999998, 2021-01-15T19:08:43.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087801389999999993, 2021-01-15T19:08:43.583 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.583 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:43.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.584 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 442 2021-01-15T19:08:43.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.584 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264015+0000", 2021-01-15T19:08:43.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264015+0000", 2021-01-15T19:08:43.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1251000000000003e-05 2021-01-15T19:08:43.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264046+0000", 2021-01-15T19:08:43.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7870000000000004e-06 2021-01-15T19:08:43.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264053+0000", 2021-01-15T19:08:43.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2762000000000001e-05 2021-01-15T19:08:43.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264065+0000", 2021-01-15T19:08:43.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1321000000000001e-05 2021-01-15T19:08:43.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264077+0000", 2021-01-15T19:08:43.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7136000000000002e-05 2021-01-15T19:08:43.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264144+0000", 2021-01-15T19:08:43.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1284000000000001e-05 2021-01-15T19:08:43.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:43.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264155+0000", 2021-01-15T19:08:43.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028618139999999999 2021-01-15T19:08:43.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267017+0000", 2021-01-15T19:08:43.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019028799999999999 2021-01-15T19:08:43.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267207+0000", 2021-01-15T19:08:43.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005587496 2021-01-15T19:08:43.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272795+0000", 2021-01-15T19:08:43.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087801389999999993 2021-01-15T19:08:43.595 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.595 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.596 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:443 3.4s0 3:2ea4cd8e:::smithi00140323-4:head [read 0~57344,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:43.596 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264114+0000", 2021-01-15T19:08:43.597 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8647739579999998, 2021-01-15T19:08:43.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095470299999999998, 2021-01-15T19:08:43.597 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.597 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:43.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.598 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 443 2021-01-15T19:08:43.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.598 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264114+0000", 2021-01-15T19:08:43.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264114+0000", 2021-01-15T19:08:43.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4766999999999999e-05 2021-01-15T19:08:43.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264169+0000", 2021-01-15T19:08:43.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6799999999999993e-06 2021-01-15T19:08:43.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264176+0000", 2021-01-15T19:08:43.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2468e-05 2021-01-15T19:08:43.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264189+0000", 2021-01-15T19:08:43.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2129999999999999e-06 2021-01-15T19:08:43.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264194+0000", 2021-01-15T19:08:43.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4391999999999999e-05 2021-01-15T19:08:43.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264228+0000", 2021-01-15T19:08:43.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5869999999999993e-06 2021-01-15T19:08:43.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:43.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264237+0000", 2021-01-15T19:08:43.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005597332 2021-01-15T19:08:43.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.269834+0000", 2021-01-15T19:08:43.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000199144 2021-01-15T19:08:43.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270033+0000", 2021-01-15T19:08:43.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036274469999999998 2021-01-15T19:08:43.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273661+0000", 2021-01-15T19:08:43.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095470299999999998 2021-01-15T19:08:43.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.611 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:444 3.4s0 3:2ea4cd8e:::smithi00140323-4:head [read 0~57344,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:43.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264235+0000", 2021-01-15T19:08:43.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.864652532, 2021-01-15T19:08:43.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096712050000000004, 2021-01-15T19:08:43.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.616 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:43.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:43.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:43.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:43.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 444 2021-01-15T19:08:43.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.617 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264235+0000", 2021-01-15T19:08:43.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264235+0000", 2021-01-15T19:08:43.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:08:43.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264276+0000", 2021-01-15T19:08:43.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3340000000000001e-06 2021-01-15T19:08:43.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264281+0000", 2021-01-15T19:08:43.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2699999999999999e-06 2021-01-15T19:08:43.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264288+0000", 2021-01-15T19:08:43.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3859999999999999e-06 2021-01-15T19:08:43.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264292+0000", 2021-01-15T19:08:43.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8819999999999998e-05 2021-01-15T19:08:43.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264330+0000", 2021-01-15T19:08:43.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7589999999999997e-06 2021-01-15T19:08:43.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:43.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264338+0000", 2021-01-15T19:08:43.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075547779999999998 2021-01-15T19:08:43.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.271893+0000", 2021-01-15T19:08:43.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123719 2021-01-15T19:08:43.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:43.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272017+0000", 2021-01-15T19:08:43.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018898999999999999 2021-01-15T19:08:43.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273907+0000", 2021-01-15T19:08:43.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096712050000000004 2021-01-15T19:08:43.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.631 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.634 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s1 51/21 ECSubWrite(tid=44, reqid=client.4418.0:445, at_version=51'18, trim_to=0'0, roll_forward_to=50'16))", 2021-01-15T19:08:43.634 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.266861+0000", 2021-01-15T19:08:43.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8620271759999998, 2021-01-15T19:08:43.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057893349999999996, 2021-01-15T19:08:43.634 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.635 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:43.635 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266861+0000", 2021-01-15T19:08:43.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266861+0000", 2021-01-15T19:08:43.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8260000000000002e-05 2021-01-15T19:08:43.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266909+0000", 2021-01-15T19:08:43.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8969999999999999e-05 2021-01-15T19:08:43.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266938+0000", 2021-01-15T19:08:43.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6042e-05 2021-01-15T19:08:43.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266954+0000", 2021-01-15T19:08:43.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.939e-06 2021-01-15T19:08:43.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266961+0000", 2021-01-15T19:08:43.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000182938 2021-01-15T19:08:43.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267144+0000", 2021-01-15T19:08:43.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2103000000000002e-05 2021-01-15T19:08:43.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267166+0000", 2021-01-15T19:08:43.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054348909999999999 2021-01-15T19:08:43.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272601+0000", 2021-01-15T19:08:43.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9191999999999999e-05 2021-01-15T19:08:43.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272650+0000", 2021-01-15T19:08:43.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057893349999999996 2021-01-15T19:08:43.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.646 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s1 51/21 ECSubWrite(tid=45, reqid=client.4418.0:446, at_version=51'19, trim_to=0'0, roll_forward_to=50'16))", 2021-01-15T19:08:43.647 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.267624+0000", 2021-01-15T19:08:43.647 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.8612642140000002, 2021-01-15T19:08:43.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005681602, 2021-01-15T19:08:43.648 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:43.648 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:43.648 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:43.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:43.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267624+0000", 2021-01-15T19:08:43.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:43.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:43.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267624+0000", 2021-01-15T19:08:43.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8810999999999997e-05 2021-01-15T19:08:43.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:43.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267672+0000", 2021-01-15T19:08:43.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1433e-05 2021-01-15T19:08:43.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:43.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267694+0000", 2021-01-15T19:08:43.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6560000000000001e-05 2021-01-15T19:08:43.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:43.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267710+0000", 2021-01-15T19:08:43.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0244000000000001e-05 2021-01-15T19:08:43.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:43.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.267721+0000", 2021-01-15T19:08:43.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025093289999999998 2021-01-15T19:08:43.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:43.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270230+0000", 2021-01-15T19:08:43.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026310800000000002 2021-01-15T19:08:43.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:43.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270493+0000", 2021-01-15T19:08:43.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027749810000000001 2021-01-15T19:08:43.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:43.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273268+0000", 2021-01-15T19:08:43.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7135999999999998e-05 2021-01-15T19:08:43.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:43.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:43.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:43.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273305+0000", 2021-01-15T19:08:43.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005681602 2021-01-15T19:08:43.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:43.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:43.660 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:43.661 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:43.483+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:43.661 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:43.663 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:43.662+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:43.764 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:43.763+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:43.797 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:43.798 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:43.798 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:43.798 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:43.805 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:43.859 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2021-01-15T19:08:43.514387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":45694976,"num_objects":62,"num_object_clones":21,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":2,"num_read":181,"num_read_kb":41572,"num_write":245,"num_write_kb":48009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":469,"ondisk_log_size":469,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":145344,"kb_used_data":93696,"kb_used_omap":0,"kb_used_meta":51584,"kb_avail":754829376,"statfs":{"total":773094113280,"available":772945281024,"internally_reserved":0,"allocated":95944704,"data_stored":121414124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":52822016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3047424,"num_objects":-6,"num_object_clones":4,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6,"num_whiteouts":1,"num_read":-35,"num_read_kb":11051,"num_write":-44,"num_write_kb":-3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000844"},"pg_stats":[{"pgid":"3.19","version":"51'26","reported_seq":"132","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.242101+0000","last_change":"2021-01-15T19:08:41.242101+0000","last_active":"2021-01-15T19:08:41.242101+0000","last_peered":"2021-01-15T19:08:41.242101+0000","last_clean":"2021-01-15T19:08:41.242101+0000","last_became_active":"2021-01-15T19:08:41.241524+0000","last_became_peered":"2021-01-15T19:08:41.241524+0000","last_unstale":"2021-01-15T19:08:41.242101+0000","last_undegraded":"2021-01-15T19:08:41.242101+0000","last_fullsized":"2021-01-15T19:08:41.242101+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2281472,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":10,"num_read_kb":1468,"num_write":10,"num_write_kb":2228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.18","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"93","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.238387+0000","last_change":"2021-01-15T19:08:41.238387+0000","last_active":"2021-01-15T19:08:41.238387+0000","last_peered":"2021-01-15T19:08:41.238387+0000","last_clean":"2021-01-15T19:08:41.238387+0000","last_became_active":"2021-01-15T19:08:41.237887+0000","last_became_peered":"2021-01-15T19:08:41.237887+0000","last_unstale":"2021-01-15T19:08:41.238387+0000","last_undegraded":"2021-01-15T19:08:41.238387+0000","last_fullsized":"2021-01-15T19:08:41.238387+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.12","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"43'30","reported_seq":"123","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.222032+0000","last_change":"2021-01-15T19:08:40.222032+0000","last_active":"2021-01-15T19:08:40.221350+0000","last_peered":"2021-01-15T19:08:39.180743+0000","last_clean":"2021-01-15T19:08:39.180743+0000","last_became_active":"2021-01-15T19:08:10.703953+0000","last_became_peered":"2021-01-15T19:08:10.703953+0000","last_unstale":"2021-01-15T19:08:40.222032+0000","last_undegraded":"2021-01-15T19:08:40.222032+0000","last_fullsized":"2021-01-15T19:08:40.222032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1445888,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":207,"num_write":8,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":"49","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.226083+0000","last_change":"2021-01-15T19:08:35.209452+0000","last_active":"2021-01-15T19:08:40.226083+0000","last_peered":"2021-01-15T19:08:40.226083+0000","last_clean":"2021-01-15T19:08:40.226083+0000","last_became_active":"2021-01-15T19:08:35.208618+0000","last_became_peered":"2021-01-15T19:08:35.208618+0000","last_unstale":"2021-01-15T19:08:40.226083+0000","last_undegraded":"2021-01-15T19:08:40.226083+0000","last_fullsized":"2021-01-15T19:08:40.226083+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"55","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.180617+0000","last_change":"2021-01-15T19:08:38.030592+0000","last_active":"2021-01-15T19:08:39.180617+0000","last_peered":"2021-01-15T19:08:39.180617+0000","last_clean":"2021-01-15T19:08:39.180617+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:08:39.180617+0000","last_undegraded":"2021-01-15T19:08:39.180617+0000","last_fullsized":"2021-01-15T19:08:39.180617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.030523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.030523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'30","reported_seq":"119","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.218960+0000","last_change":"2021-01-15T19:08:40.218960+0000","last_active":"2021-01-15T19:08:40.217952+0000","last_peered":"2021-01-15T19:08:39.180926+0000","last_clean":"2021-01-15T19:08:39.180926+0000","last_became_active":"2021-01-15T19:08:10.703898+0000","last_became_peered":"2021-01-15T19:08:10.703898+0000","last_unstale":"2021-01-15T19:08:40.218960+0000","last_undegraded":"2021-01-15T19:08:40.218960+0000","last_fullsized":"2021-01-15T19:08:40.218960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1519616,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":7,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.4","version":"50'33","reported_seq":"129","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.222498+0000","last_change":"2021-01-15T19:08:40.222498+0000","last_active":"2021-01-15T19:08:40.220471+0000","last_peered":"2021-01-15T19:08:39.273867+0000","last_clean":"2021-01-15T19:08:39.273867+0000","last_became_active":"2021-01-15T19:08:10.228436+0000","last_became_peered":"2021-01-15T19:08:10.228436+0000","last_unstale":"2021-01-15T19:08:40.222498+0000","last_undegraded":"2021-01-15T19:08:40.222498+0000","last_fullsized":"2021-01-15T19:08:40.222498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2433024,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4023,"num_write":8,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":"49","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225803+0000","last_change":"2021-01-15T19:08:35.215835+0000","last_active":"2021-01-15T19:08:40.225803+0000","last_peered":"2021-01-15T19:08:40.225803+0000","last_clean":"2021-01-15T19:08:40.225803+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:08:40.225803+0000","last_undegraded":"2021-01-15T19:08:40.225803+0000","last_fullsized":"2021-01-15T19:08:40.225803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"43'35","reported_seq":"125","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.218743+0000","last_change":"2021-01-15T19:08:40.218743+0000","last_active":"2021-01-15T19:08:40.217857+0000","last_peered":"2021-01-15T19:08:39.266406+0000","last_clean":"2021-01-15T19:08:39.266406+0000","last_became_active":"2021-01-15T19:08:10.230297+0000","last_became_peered":"2021-01-15T19:08:10.230297+0000","last_unstale":"2021-01-15T19:08:40.218743+0000","last_undegraded":"2021-01-15T19:08:40.218743+0000","last_fullsized":"2021-01-15T19:08:40.218743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2998272,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":658,"num_write":19,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":"49","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225043+0000","last_change":"2021-01-15T19:08:35.211611+0000","last_active":"2021-01-15T19:08:40.225043+0000","last_peered":"2021-01-15T19:08:40.225043+0000","last_clean":"2021-01-15T19:08:40.225043+0000","last_became_active":"2021-01-15T19:08:35.210980+0000","last_became_peered":"2021-01-15T19:08:35.210980+0000","last_unstale":"2021-01-15T19:08:40.225043+0000","last_undegraded":"2021-01-15T19:08:40.225043+0000","last_fullsized":"2021-01-15T19:08:40.225043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"111","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.218190+0000","last_change":"2021-01-15T19:08:40.219095+0000","last_active":"2021-01-15T19:08:40.218190+0000","last_peered":"2021-01-15T19:08:39.192593+0000","last_clean":"2021-01-15T19:08:39.192593+0000","last_became_active":"2021-01-15T19:08:10.234192+0000","last_became_peered":"2021-01-15T19:08:10.234192+0000","last_unstale":"2021-01-15T19:08:40.218190+0000","last_undegraded":"2021-01-15T19:08:40.218190+0000","last_fullsized":"2021-01-15T19:08:40.218190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":987136,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1610,"num_write":5,"num_write_kb":964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"1.0","version":"14'2","reported_seq":"48","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.238413+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:39.238413+0000","last_peered":"2021-01-15T19:08:39.238413+0000","last_clean":"2021-01-15T19:08:39.238413+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:39.238413+0000","last_undegraded":"2021-01-15T19:08:39.238413+0000","last_fullsized":"2021-01-15T19:08:39.238413+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"49","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225637+0000","last_change":"2021-01-15T19:08:35.211897+0000","last_active":"2021-01-15T19:08:40.225637+0000","last_peered":"2021-01-15T19:08:40.225637+0000","last_clean":"2021-01-15T19:08:40.225637+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:08:40.225637+0000","last_undegraded":"2021-01-15T19:08:40.225637+0000","last_fullsized":"2021-01-15T19:08:40.225637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"51'61","reported_seq":"161","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.231141+0000","last_change":"2021-01-15T19:08:40.231141+0000","last_active":"2021-01-15T19:08:40.230016+0000","last_peered":"2021-01-15T19:08:39.259504+0000","last_clean":"2021-01-15T19:08:39.259504+0000","last_became_active":"2021-01-15T19:08:10.231211+0000","last_became_peered":"2021-01-15T19:08:10.231211+0000","last_unstale":"2021-01-15T19:08:40.231141+0000","last_undegraded":"2021-01-15T19:08:40.231141+0000","last_fullsized":"2021-01-15T19:08:40.231141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":61,"ondisk_log_size":61,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3588096,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":966,"num_write":17,"num_write_kb":4624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":"64","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225469+0000","last_change":"2021-01-15T19:08:38.923608+0000","last_active":"2021-01-15T19:08:40.225469+0000","last_peered":"2021-01-15T19:08:40.225469+0000","last_clean":"2021-01-15T19:08:40.225469+0000","last_became_active":"2021-01-15T19:08:35.204089+0000","last_became_peered":"2021-01-15T19:08:35.204089+0000","last_unstale":"2021-01-15T19:08:40.225469+0000","last_undegraded":"2021-01-15T19:08:40.225469+0000","last_fullsized":"2021-01-15T19:08:40.225469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.923498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.923498+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"94","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.228839+0000","last_change":"2021-01-15T19:08:41.228839+0000","last_active":"2021-01-15T19:08:41.228839+0000","last_peered":"2021-01-15T19:08:41.228839+0000","last_clean":"2021-01-15T19:08:41.228839+0000","last_became_active":"2021-01-15T19:08:41.228264+0000","last_became_peered":"2021-01-15T19:08:41.228264+0000","last_unstale":"2021-01-15T19:08:41.228839+0000","last_undegraded":"2021-01-15T19:08:41.228839+0000","last_fullsized":"2021-01-15T19:08:41.228839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":"49","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225035+0000","last_change":"2021-01-15T19:08:35.214301+0000","last_active":"2021-01-15T19:08:40.225035+0000","last_peered":"2021-01-15T19:08:40.225035+0000","last_clean":"2021-01-15T19:08:40.225035+0000","last_became_active":"2021-01-15T19:08:35.213894+0000","last_became_peered":"2021-01-15T19:08:35.213894+0000","last_unstale":"2021-01-15T19:08:40.225035+0000","last_undegraded":"2021-01-15T19:08:40.225035+0000","last_fullsized":"2021-01-15T19:08:40.225035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"51'20","reported_seq":"107","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.233304+0000","last_change":"2021-01-15T19:08:40.233304+0000","last_active":"2021-01-15T19:08:40.232559+0000","last_peered":"2021-01-15T19:08:39.277745+0000","last_clean":"2021-01-15T19:08:39.277745+0000","last_became_active":"2021-01-15T19:08:10.230610+0000","last_became_peered":"2021-01-15T19:08:10.230610+0000","last_unstale":"2021-01-15T19:08:40.233304+0000","last_undegraded":"2021-01-15T19:08:40.233304+0000","last_fullsized":"2021-01-15T19:08:40.233304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":774144,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":15,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":"53","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.732649+0000","last_change":"2021-01-15T19:08:39.732649+0000","last_active":"2021-01-15T19:08:39.732649+0000","last_peered":"2021-01-15T19:08:39.732649+0000","last_clean":"2021-01-15T19:08:39.732649+0000","last_became_active":"2021-01-15T19:08:35.204979+0000","last_became_peered":"2021-01-15T19:08:35.204979+0000","last_unstale":"2021-01-15T19:08:39.732649+0000","last_undegraded":"2021-01-15T19:08:39.732649+0000","last_fullsized":"2021-01-15T19:08:39.732649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:39.732565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:39.732565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"51'34","reported_seq":"130","reported_epoch":"52","state":"peering","last_fresh":"2021-01-15T19:08:40.218902+0000","last_change":"2021-01-15T19:08:40.218902+0000","last_active":"2021-01-15T19:08:40.217927+0000","last_peered":"2021-01-15T19:08:39.285930+0000","last_clean":"2021-01-15T19:08:39.285930+0000","last_became_active":"2021-01-15T19:08:10.231245+0000","last_became_peered":"2021-01-15T19:08:10.231245+0000","last_unstale":"2021-01-15T19:08:40.218902+0000","last_undegraded":"2021-01-15T19:08:40.218902+0000","last_fullsized":"2021-01-15T19:08:40.218902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3014656,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":304,"num_write":10,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.9","version":"0'0","reported_seq":"12","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.180409+0000","last_change":"2021-01-15T19:08:37.215636+0000","last_active":"2021-01-15T19:08:39.180409+0000","last_peered":"2021-01-15T19:08:39.180409+0000","last_clean":"2021-01-15T19:08:39.180409+0000","last_became_active":"2021-01-15T19:08:37.215228+0000","last_became_peered":"2021-01-15T19:08:37.215228+0000","last_unstale":"2021-01-15T19:08:39.180409+0000","last_undegraded":"2021-01-15T19:08:39.180409+0000","last_fullsized":"2021-01-15T19:08:39.180409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"51'26","reported_seq":"133","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.238890+0000","last_change":"2021-01-15T19:08:41.238890+0000","last_active":"2021-01-15T19:08:41.238890+0000","last_peered":"2021-01-15T19:08:41.238890+0000","last_clean":"2021-01-15T19:08:41.238890+0000","last_became_active":"2021-01-15T19:08:41.238249+0000","last_became_peered":"2021-01-15T19:08:41.238249+0000","last_unstale":"2021-01-15T19:08:41.238890+0000","last_undegraded":"2021-01-15T19:08:41.238890+0000","last_fullsized":"2021-01-15T19:08:41.238890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2281472,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":1468,"num_write":9,"num_write_kb":2228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":"13","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.228729+0000","last_change":"2021-01-15T19:08:37.217407+0000","last_active":"2021-01-15T19:08:40.228729+0000","last_peered":"2021-01-15T19:08:40.228729+0000","last_clean":"2021-01-15T19:08:40.228729+0000","last_became_active":"2021-01-15T19:08:37.216976+0000","last_became_peered":"2021-01-15T19:08:37.216976+0000","last_unstale":"2021-01-15T19:08:40.228729+0000","last_undegraded":"2021-01-15T19:08:40.228729+0000","last_fullsized":"2021-01-15T19:08:40.228729+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"52'39","reported_seq":"133","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.282519+0000","last_change":"2021-01-15T19:08:39.016834+0000","last_active":"2021-01-15T19:08:41.282519+0000","last_peered":"2021-01-15T19:08:41.282519+0000","last_clean":"2021-01-15T19:08:41.282519+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:41.282519+0000","last_undegraded":"2021-01-15T19:08:41.282519+0000","last_fullsized":"2021-01-15T19:08:41.282519+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8306688,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":20,"num_read_kb":14746,"num_write":25,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.b","version":"0'0","reported_seq":"48","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225572+0000","last_change":"2021-01-15T19:08:35.210579+0000","last_active":"2021-01-15T19:08:40.225572+0000","last_peered":"2021-01-15T19:08:40.225572+0000","last_clean":"2021-01-15T19:08:40.225572+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:08:40.225572+0000","last_undegraded":"2021-01-15T19:08:40.225572+0000","last_fullsized":"2021-01-15T19:08:40.225572+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"50'11","reported_seq":"110","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.225514+0000","last_change":"2021-01-15T19:08:38.683456+0000","last_active":"2021-01-15T19:08:40.225514+0000","last_peered":"2021-01-15T19:08:40.225514+0000","last_clean":"2021-01-15T19:08:40.225514+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:40.225514+0000","last_undegraded":"2021-01-15T19:08:40.225514+0000","last_fullsized":"2021-01-15T19:08:40.225514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2621440,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":9822,"num_write":8,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.a","version":"0'0","reported_seq":"52","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:40.178279+0000","last_change":"2021-01-15T19:08:40.178279+0000","last_active":"2021-01-15T19:08:40.178279+0000","last_peered":"2021-01-15T19:08:40.178279+0000","last_clean":"2021-01-15T19:08:40.178279+0000","last_became_active":"2021-01-15T19:08:37.218600+0000","last_became_peered":"2021-01-15T19:08:37.218600+0000","last_unstale":"2021-01-15T19:08:40.178279+0000","last_undegraded":"2021-01-15T19:08:40.178279+0000","last_fullsized":"2021-01-15T19:08:40.178279+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.178185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.178185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"53'52","reported_seq":"146","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.262475+0000","last_change":"2021-01-15T19:08:38.179554+0000","last_active":"2021-01-15T19:08:41.262475+0000","last_peered":"2021-01-15T19:08:41.262475+0000","last_clean":"2021-01-15T19:08:41.262475+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:41.262475+0000","last_undegraded":"2021-01-15T19:08:41.262475+0000","last_fullsized":"2021-01-15T19:08:41.262475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7815168,"num_objects":10,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":26,"num_read_kb":2609,"num_write":37,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.d","version":"0'0","reported_seq":"12","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.183813+0000","last_change":"2021-01-15T19:08:37.216593+0000","last_active":"2021-01-15T19:08:39.183813+0000","last_peered":"2021-01-15T19:08:39.183813+0000","last_clean":"2021-01-15T19:08:39.183813+0000","last_became_active":"2021-01-15T19:08:37.216171+0000","last_became_peered":"2021-01-15T19:08:37.216171+0000","last_unstale":"2021-01-15T19:08:39.183813+0000","last_undegraded":"2021-01-15T19:08:39.183813+0000","last_fullsized":"2021-01-15T19:08:39.183813+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"51'14","reported_seq":"113","reported_epoch":"53","state":"active+clean","last_fresh":"2021-01-15T19:08:41.266720+0000","last_change":"2021-01-15T19:08:39.014884+0000","last_active":"2021-01-15T19:08:41.266720+0000","last_peered":"2021-01-15T19:08:41.266720+0000","last_clean":"2021-01-15T19:08:41.266720+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:41.266720+0000","last_undegraded":"2021-01-15T19:08:41.266720+0000","last_fullsized":"2021-01-15T19:08:41.266720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1097728,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":510,"num_write":27,"num_write_kb":1072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.c","version":"0'0","reported_seq":"49","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.266695+0000","last_change":"2021-01-15T19:08:37.216670+0000","last_active":"2021-01-15T19:08:39.266695+0000","last_peered":"2021-01-15T19:08:39.266695+0000","last_clean":"2021-01-15T19:08:39.266695+0000","last_became_active":"2021-01-15T19:08:37.216240+0000","last_became_peered":"2021-01-15T19:08:37.216240+0000","last_unstale":"2021-01-15T19:08:39.266695+0000","last_undegraded":"2021-01-15T19:08:39.266695+0000","last_fullsized":"2021-01-15T19:08:39.266695+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"39'8","reported_seq":"100","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.180458+0000","last_change":"2021-01-15T19:08:38.183423+0000","last_active":"2021-01-15T19:08:39.180458+0000","last_peered":"2021-01-15T19:08:39.180458+0000","last_clean":"2021-01-15T19:08:39.180458+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:39.180458+0000","last_undegraded":"2021-01-15T19:08:39.180458+0000","last_fullsized":"2021-01-15T19:08:39.180458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":753664,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":555,"num_write":5,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.f","version":"0'0","reported_seq":"53","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.228139+0000","last_change":"2021-01-15T19:08:37.964979+0000","last_active":"2021-01-15T19:08:40.228139+0000","last_peered":"2021-01-15T19:08:40.228139+0000","last_clean":"2021-01-15T19:08:40.228139+0000","last_became_active":"2021-01-15T19:08:37.224346+0000","last_became_peered":"2021-01-15T19:08:37.224346+0000","last_unstale":"2021-01-15T19:08:40.228139+0000","last_undegraded":"2021-01-15T19:08:40.228139+0000","last_fullsized":"2021-01-15T19:08:40.228139+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"40'30","reported_seq":"123","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.228722+0000","last_change":"2021-01-15T19:08:38.179900+0000","last_active":"2021-01-15T19:08:40.228722+0000","last_peered":"2021-01-15T19:08:40.228722+0000","last_clean":"2021-01-15T19:08:40.228722+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:40.228722+0000","last_undegraded":"2021-01-15T19:08:40.228722+0000","last_fullsized":"2021-01-15T19:08:40.228722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3776512,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":2623,"num_write":33,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.e","version":"0'0","reported_seq":"58","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.180667+0000","last_change":"2021-01-15T19:08:37.216877+0000","last_active":"2021-01-15T19:08:39.180667+0000","last_peered":"2021-01-15T19:08:39.180667+0000","last_clean":"2021-01-15T19:08:39.180667+0000","last_became_active":"2021-01-15T19:08:37.216423+0000","last_became_peered":"2021-01-15T19:08:37.216423+0000","last_unstale":"2021-01-15T19:08:39.180667+0000","last_undegraded":"2021-01-15T19:08:39.180667+0000","last_fullsized":"2021-01-15T19:08:39.180667+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":"12","reported_epoch":"51","state":"active+clean","last_fresh":"2021-01-15T19:08:39.180352+0000","last_change":"2021-01-15T19:08:37.214774+0000","last_active":"2021-01-15T19:08:39.180352+0000","last_peered":"2021-01-15T19:08:39.180352+0000","last_clean":"2021-01-15T19:08:39.180352+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:08:39.180352+0000","last_undegraded":"2021-01-15T19:08:39.180352+0000","last_fullsized":"2021-01-15T19:08:39.180352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"13","reported_epoch":"52","state":"active+clean","last_fresh":"2021-01-15T19:08:40.230087+0000","last_change":"2021-01-15T19:08:37.224434+0000","last_active":"2021-01-15T19:08:40.230087+0000","last_peered":"2021-01-15T19:08:40.230087+0000","last_clean":"2021-01-15T19:08:40.230087+0000","last_became_active":"2021-01-15T19:08:37.223920+0000","last_became_peered":"2021-01-15T19:08:37.223920+0000","last_unstale":"2021-01-15T19:08:40.230087+0000","last_undegraded":"2021-01-15T19:08:40.230087+0000","last_fullsized":"2021-01-15T19:08:40.230087+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:40.209770+0000","last_change":"2021-01-15T19:08:40.209770+0000","last_active":"2021-01-15T19:08:40.209770+0000","last_peered":"2021-01-15T19:08:40.209770+0000","last_clean":"2021-01-15T19:08:40.209770+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:40.209770+0000","last_undegraded":"2021-01-15T19:08:40.209770+0000","last_fullsized":"2021-01-15T19:08:40.209770+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:40.209770+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.209770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":45694976,"num_objects":60,"num_object_clones":21,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":2,"num_read":181,"num_read_kb":41572,"num_write":243,"num_write_kb":48000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":93093888,"data_stored":121565184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":467,"ondisk_log_size":467,"up":54,"acting":54,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23684,"kb_used_data":17596,"kb_used_omap":0,"kb_used_meta":6080,"kb_avail":94348156,"statfs":{"total":96636764160,"available":96612511744,"internally_reserved":0,"allocated":18018304,"data_stored":23692366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6225920},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15628,"kb_used_data":8836,"kb_used_omap":0,"kb_used_meta":6784,"kb_avail":94356212,"statfs":{"total":96636764160,"available":96620761088,"internally_reserved":0,"allocated":9048064,"data_stored":11250330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6946816},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15460,"kb_used_data":9180,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":94356380,"statfs":{"total":96636764160,"available":96620933120,"internally_reserved":0,"allocated":9400320,"data_stored":13240986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6739999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15008,"kb_used_data":8856,"kb_used_omap":0,"kb_used_meta":6144,"kb_avail":94356832,"statfs":{"total":96636764160,"available":96621395968,"internally_reserved":0,"allocated":9068544,"data_stored":11428942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6291456},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":20752,"kb_used_data":13704,"kb_used_omap":0,"kb_used_meta":7040,"kb_avail":94351088,"statfs":{"total":96636764160,"available":96615514112,"internally_reserved":0,"allocated":14032896,"data_stored":16960154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7208960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":16988,"kb_used_data":11220,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":94354852,"statfs":{"total":96636764160,"available":96619368448,"internally_reserved":0,"allocated":11489280,"data_stored":15389774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":25428,"kb_used_data":17484,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94346412,"statfs":{"total":96636764160,"available":96610725888,"internally_reserved":0,"allocated":17903616,"data_stored":21805722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12396,"kb_used_data":6820,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":94359444,"statfs":{"total":96636764160,"available":96624070656,"internally_reserved":0,"allocated":6983680,"data_stored":7645850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]}]}} 2021-01-15T19:08:43.865 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2021-01-15T19:08:43.865 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 11 2021-01-15T19:08:43.867 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:43.866+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:43.971 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:43.972 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:43.972 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:43.972 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:43.972 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:43.973 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:43.970+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:43.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:44.072 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:44.071+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:44.151 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:44.151 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:44.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:44.151 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:44.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.152 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=87, reqid=client.4418.0:313, at_version=41'14, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:44.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:44.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.940603088, 2021-01-15T19:08:44.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998, 2021-01-15T19:08:44.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.154 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.154 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:44.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:44.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0902000000000002e-05 2021-01-15T19:08:44.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204662+0000", 2021-01-15T19:08:44.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8193e-05 2021-01-15T19:08:44.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204680+0000", 2021-01-15T19:08:44.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958e-05 2021-01-15T19:08:44.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204698+0000", 2021-01-15T19:08:44.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2021-01-15T19:08:44.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204706+0000", 2021-01-15T19:08:44.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2994000000000002e-05 2021-01-15T19:08:44.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204749+0000", 2021-01-15T19:08:44.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491999999999999e-05 2021-01-15T19:08:44.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204760+0000", 2021-01-15T19:08:44.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056169770000000004 2021-01-15T19:08:44.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210377+0000", 2021-01-15T19:08:44.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8433999999999997e-05 2021-01-15T19:08:44.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210425+0000", 2021-01-15T19:08:44.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998 2021-01-15T19:08:44.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:44.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:44.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.921614663, 2021-01-15T19:08:44.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:44.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.173 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:44.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:44.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:44.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:44.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:44.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:44.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:44.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:44.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:44.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:44.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:44.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:44.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:44.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:44.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:44.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:44.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:44.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:44.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:44.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.186 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=44, reqid=client.4418.0:334, at_version=41'25, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:44.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:44.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.919920446000001, 2021-01-15T19:08:44.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001, 2021-01-15T19:08:44.187 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:44.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:44.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9133000000000004e-05 2021-01-15T19:08:44.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225363+0000", 2021-01-15T19:08:44.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2021-01-15T19:08:44.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225376+0000", 2021-01-15T19:08:44.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0949e-05 2021-01-15T19:08:44.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225387+0000", 2021-01-15T19:08:44.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7710000000000001e-06 2021-01-15T19:08:44.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225394+0000", 2021-01-15T19:08:44.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5732999999999997e-05 2021-01-15T19:08:44.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225440+0000", 2021-01-15T19:08:44.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5984000000000001e-05 2021-01-15T19:08:44.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225456+0000", 2021-01-15T19:08:44.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059787429999999999 2021-01-15T19:08:44.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231434+0000", 2021-01-15T19:08:44.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2653000000000001e-05 2021-01-15T19:08:44.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231457+0000", 2021-01-15T19:08:44.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001 2021-01-15T19:08:44.203 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.203 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.203 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.204 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:340 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~729088,checksum] snapc 0=[] ondisk+read+known_if_redirected e42)", 2021-01-15T19:08:44.204 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:44.204 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.965240166999999, 2021-01-15T19:08:44.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056637780000000004, 2021-01-15T19:08:44.204 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.205 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:44.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:44.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:44.205 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:44.205 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 340 2021-01-15T19:08:44.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.207 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:44.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.179974+0000", 2021-01-15T19:08:44.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6628000000000001e-05 2021-01-15T19:08:44.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180030+0000", 2021-01-15T19:08:44.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:08:44.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180038+0000", 2021-01-15T19:08:44.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3541e-05 2021-01-15T19:08:44.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180051+0000", 2021-01-15T19:08:44.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:08:44.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180056+0000", 2021-01-15T19:08:44.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3109999999999998e-05 2021-01-15T19:08:44.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180110+0000", 2021-01-15T19:08:44.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9489999999999996e-06 2021-01-15T19:08:44.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:44.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.180118+0000", 2021-01-15T19:08:44.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033250100000000002 2021-01-15T19:08:44.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.183443+0000", 2021-01-15T19:08:44.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2630000000000002e-05 2021-01-15T19:08:44.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:44.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.183505+0000", 2021-01-15T19:08:44.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021323370000000002 2021-01-15T19:08:44.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:30.185638+0000", 2021-01-15T19:08:44.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056637780000000004 2021-01-15T19:08:44.221 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.221 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.221 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.222 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 47/21 ECSubWrite(tid=101, reqid=client.4418.0:384, at_version=47'23, trim_to=0'0, roll_forward_to=44'21))", 2021-01-15T19:08:44.223 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:44.223 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.904093348, 2021-01-15T19:08:44.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002, 2021-01-15T19:08:44.223 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.224 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.224 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:44.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029587999999999999 2021-01-15T19:08:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241416+0000", 2021-01-15T19:08:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6582000000000001e-05 2021-01-15T19:08:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241433+0000", 2021-01-15T19:08:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2021-01-15T19:08:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241444+0000", 2021-01-15T19:08:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.63e-06 2021-01-15T19:08:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241451+0000", 2021-01-15T19:08:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6316999999999997e-05 2021-01-15T19:08:44.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241497+0000", 2021-01-15T19:08:44.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241506+0000", 2021-01-15T19:08:44.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058582230000000001 2021-01-15T19:08:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247365+0000", 2021-01-15T19:08:44.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6705e-05 2021-01-15T19:08:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247421+0000", 2021-01-15T19:08:44.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002 2021-01-15T19:08:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=50'50, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9638750270000003, 2021-01-15T19:08:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996, 2021-01-15T19:08:44.236 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9509999999999999e-05 2021-01-15T19:08:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181388+0000", 2021-01-15T19:08:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.141e-05 2021-01-15T19:08:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181410+0000", 2021-01-15T19:08:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5201000000000001e-05 2021-01-15T19:08:44.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181425+0000", 2021-01-15T19:08:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5440000000000004e-06 2021-01-15T19:08:44.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181432+0000", 2021-01-15T19:08:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029235099999999999 2021-01-15T19:08:44.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181724+0000", 2021-01-15T19:08:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1324e-05 2021-01-15T19:08:44.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181735+0000", 2021-01-15T19:08:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005959865 2021-01-15T19:08:44.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187695+0000", 2021-01-15T19:08:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4237999999999999e-05 2021-01-15T19:08:44.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187739+0000", 2021-01-15T19:08:44.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996 2021-01-15T19:08:44.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=73, reqid=osd.5.0:73, at_version=50'52, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9634613459999999, 2021-01-15T19:08:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003, 2021-01-15T19:08:44.250 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7224000000000001e-05 2021-01-15T19:08:44.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181800+0000", 2021-01-15T19:08:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5681999999999999e-05 2021-01-15T19:08:44.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181825+0000", 2021-01-15T19:08:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5474000000000001e-05 2021-01-15T19:08:44.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181841+0000", 2021-01-15T19:08:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8179999999999993e-06 2021-01-15T19:08:44.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181850+0000", 2021-01-15T19:08:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150703 2021-01-15T19:08:44.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182000+0000", 2021-01-15T19:08:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2021-01-15T19:08:44.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182010+0000", 2021-01-15T19:08:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066268780000000001 2021-01-15T19:08:44.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188637+0000", 2021-01-15T19:08:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6959000000000002e-05 2021-01-15T19:08:44.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188684+0000", 2021-01-15T19:08:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003 2021-01-15T19:08:44.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=152, reqid=osd.4.0:152, at_version=50'30, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:44.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9604446930000003, 2021-01-15T19:08:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998, 2021-01-15T19:08:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1204e-05 2021-01-15T19:08:44.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184820+0000", 2021-01-15T19:08:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1126000000000001e-05 2021-01-15T19:08:44.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184842+0000", 2021-01-15T19:08:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5197999999999999e-05 2021-01-15T19:08:44.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184857+0000", 2021-01-15T19:08:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.336e-06 2021-01-15T19:08:44.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184863+0000", 2021-01-15T19:08:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2200000000000002e-05 2021-01-15T19:08:44.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184915+0000", 2021-01-15T19:08:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999e-05 2021-01-15T19:08:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184927+0000", 2021-01-15T19:08:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065179089999999997 2021-01-15T19:08:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191445+0000", 2021-01-15T19:08:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2212999999999997e-05 2021-01-15T19:08:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191487+0000", 2021-01-15T19:08:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998 2021-01-15T19:08:44.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.276 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=153, reqid=osd.4.0:153, at_version=50'32, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9603023439999996, 2021-01-15T19:08:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999, 2021-01-15T19:08:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.277 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:44.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:44.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0540000000000001e-05 2021-01-15T19:08:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184962+0000", 2021-01-15T19:08:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1452999999999999e-05 2021-01-15T19:08:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184984+0000", 2021-01-15T19:08:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4532000000000001e-05 2021-01-15T19:08:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184998+0000", 2021-01-15T19:08:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9069999999999998e-06 2021-01-15T19:08:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.185003+0000", 2021-01-15T19:08:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027679219999999999 2021-01-15T19:08:44.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187771+0000", 2021-01-15T19:08:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1089e-05 2021-01-15T19:08:44.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187782+0000", 2021-01-15T19:08:44.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004853792 2021-01-15T19:08:44.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192636+0000", 2021-01-15T19:08:44.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0936000000000003e-05 2021-01-15T19:08:44.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192667+0000", 2021-01-15T19:08:44.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999 2021-01-15T19:08:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:08:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9604926310000002, 2021-01-15T19:08:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:08:44.291 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:08:44.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:08:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:08:44.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:08:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:44.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:08:44.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:08:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:08:44.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:08:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:08:44.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:08:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:08:44.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:08:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:08:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:08:44.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:08:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:08:44.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:439 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:44.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8812032759999999, 2021-01-15T19:08:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007, 2021-01-15T19:08:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:44.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2021-01-15T19:08:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6384000000000001e-05 2021-01-15T19:08:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264067+0000", 2021-01-15T19:08:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7919999999999998e-06 2021-01-15T19:08:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264076+0000", 2021-01-15T19:08:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2021-01-15T19:08:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264092+0000", 2021-01-15T19:08:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7889999999999995e-06 2021-01-15T19:08:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264100+0000", 2021-01-15T19:08:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3668999999999997e-05 2021-01-15T19:08:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264154+0000", 2021-01-15T19:08:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010867800000000001 2021-01-15T19:08:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264263+0000", 2021-01-15T19:08:44.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008062049 2021-01-15T19:08:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272325+0000", 2021-01-15T19:08:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007 2021-01-15T19:08:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:440 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:44.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8810656229999996, 2021-01-15T19:08:44.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695, 2021-01-15T19:08:44.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:44.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:44.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:44.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:44.319 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 440 2021-01-15T19:08:44.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.319 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:44.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1341000000000002e-05 2021-01-15T19:08:44.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264200+0000", 2021-01-15T19:08:44.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8589999999999999e-06 2021-01-15T19:08:44.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264206+0000", 2021-01-15T19:08:44.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1953000000000001e-05 2021-01-15T19:08:44.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264218+0000", 2021-01-15T19:08:44.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.276e-06 2021-01-15T19:08:44.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264224+0000", 2021-01-15T19:08:44.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000377647 2021-01-15T19:08:44.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264601+0000", 2021-01-15T19:08:44.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8468999999999996e-05 2021-01-15T19:08:44.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:44.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264690+0000", 2021-01-15T19:08:44.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095141500000000007 2021-01-15T19:08:44.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274204+0000", 2021-01-15T19:08:44.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695 2021-01-15T19:08:44.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.329 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.329 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:441 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:44.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:44.330 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8809451639999999, 2021-01-15T19:08:44.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001, 2021-01-15T19:08:44.330 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.331 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:44.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:44.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:44.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:44.331 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 441 2021-01-15T19:08:44.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.332 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:44.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:44.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0819000000000001e-05 2021-01-15T19:08:44.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264340+0000", 2021-01-15T19:08:44.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3950000000000001e-06 2021-01-15T19:08:44.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264348+0000", 2021-01-15T19:08:44.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2213e-05 2021-01-15T19:08:44.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264360+0000", 2021-01-15T19:08:44.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6470000000000001e-06 2021-01-15T19:08:44.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264367+0000", 2021-01-15T19:08:44.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041851429999999997 2021-01-15T19:08:44.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268552+0000", 2021-01-15T19:08:44.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108403 2021-01-15T19:08:44.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:44.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268660+0000", 2021-01-15T19:08:44.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006812706 2021-01-15T19:08:44.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275473+0000", 2021-01-15T19:08:44.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001 2021-01-15T19:08:44.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.342 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:44.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:44.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8795351809999996, 2021-01-15T19:08:44.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:08:44.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.344 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.344 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:44.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:44.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:08:44.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:08:44.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:08:44.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:08:44.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:08:44.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:08:44.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:08:44.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:08:44.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:08:44.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:08:44.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:08:44.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:08:44.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:08:44.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:08:44.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:08:44.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:08:44.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:08:44.354 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.355 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:44.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:44.356 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8751239069999999, 2021-01-15T19:08:44.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:08:44.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.357 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:44.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:44.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:08:44.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:08:44.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:08:44.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:08:44.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:08:44.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:08:44.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:08:44.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:08:44.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:08:44.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:08:44.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:08:44.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:08:44.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:08:44.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:08:44.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:08:44.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:08:44.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:08:44.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.368 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=106, reqid=client.4418.0:436, at_version=51'42, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:44.368 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:44.368 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8720157630000003, 2021-01-15T19:08:44.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049, 2021-01-15T19:08:44.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.370 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:44.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:44.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184005 2021-01-15T19:08:44.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273382+0000", 2021-01-15T19:08:44.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3705999999999999e-05 2021-01-15T19:08:44.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273416+0000", 2021-01-15T19:08:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1336999999999999e-05 2021-01-15T19:08:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273437+0000", 2021-01-15T19:08:44.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4830000000000005e-06 2021-01-15T19:08:44.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273447+0000", 2021-01-15T19:08:44.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001793602 2021-01-15T19:08:44.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275240+0000", 2021-01-15T19:08:44.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3903999999999999e-05 2021-01-15T19:08:44.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275254+0000", 2021-01-15T19:08:44.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071874139999999996 2021-01-15T19:08:44.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282442+0000", 2021-01-15T19:08:44.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7597999999999999e-05 2021-01-15T19:08:44.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282469+0000", 2021-01-15T19:08:44.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049 2021-01-15T19:08:44.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.381 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.381 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=107, reqid=client.4418.0:437, at_version=51'43, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:44.382 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:44.382 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8708510460000003, 2021-01-15T19:08:44.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003, 2021-01-15T19:08:44.382 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.382 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.383 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:44.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:44.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9589999999999998e-05 2021-01-15T19:08:44.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274412+0000", 2021-01-15T19:08:44.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0508000000000001e-05 2021-01-15T19:08:44.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274433+0000", 2021-01-15T19:08:44.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7353000000000001e-05 2021-01-15T19:08:44.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274450+0000", 2021-01-15T19:08:44.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0870000000000001e-06 2021-01-15T19:08:44.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274457+0000", 2021-01-15T19:08:44.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027924999999999998 2021-01-15T19:08:44.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277250+0000", 2021-01-15T19:08:44.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3828e-05 2021-01-15T19:08:44.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277264+0000", 2021-01-15T19:08:44.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052077039999999996 2021-01-15T19:08:44.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282471+0000", 2021-01-15T19:08:44.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3986999999999999e-05 2021-01-15T19:08:44.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282495+0000", 2021-01-15T19:08:44.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003 2021-01-15T19:08:44.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.393 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.394 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 51/21 ECSubWrite(tid=159, reqid=client.4418.0:448, at_version=51'34, trim_to=0'0, roll_forward_to=50'32))", 2021-01-15T19:08:44.395 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:44.395 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.868600174, 2021-01-15T19:08:44.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568, 2021-01-15T19:08:44.395 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.395 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:44.396 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:44.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:44.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132867 2021-01-15T19:08:44.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276747+0000", 2021-01-15T19:08:44.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6607000000000003e-05 2021-01-15T19:08:44.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276793+0000", 2021-01-15T19:08:44.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9687e-05 2021-01-15T19:08:44.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276813+0000", 2021-01-15T19:08:44.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.374e-06 2021-01-15T19:08:44.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276820+0000", 2021-01-15T19:08:44.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6462999999999998e-05 2021-01-15T19:08:44.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276877+0000", 2021-01-15T19:08:44.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6032000000000002e-05 2021-01-15T19:08:44.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276893+0000", 2021-01-15T19:08:44.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006644418 2021-01-15T19:08:44.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283537+0000", 2021-01-15T19:08:44.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5120000000000003e-05 2021-01-15T19:08:44.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283572+0000", 2021-01-15T19:08:44.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568 2021-01-15T19:08:44.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.406 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.407 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:08:44.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:44.408 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.91300445399999997, 2021-01-15T19:08:44.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:08:44.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:44.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:44.409 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:44.409 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:44.409 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:08:44.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.409 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:44.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:44.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:08:44.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:08:44.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:08:44.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:08:44.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:08:44.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:08:44.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:08:44.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:08:44.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:08:44.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:08:44.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:08:44.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:44.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:08:44.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:08:44.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:44.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:08:44.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:08:44.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:44.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:08:44.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:08:44.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:44.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:08:44.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:08:44.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:08:44.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:08:44.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:08:44.425 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:44.425 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.91285643599999999, 2021-01-15T19:08:44.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:08:44.425 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:44.425 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:44.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:44.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:44.426 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:44.426 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:08:44.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:44.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:44.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:44.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:44.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:44.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:44.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:44.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:44.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:08:44.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:08:44.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:44.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:08:44.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:08:44.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:44.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:08:44.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:08:44.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:44.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:08:44.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:08:44.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:08:44.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:08:44.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:44.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:08:44.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:08:44.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:08:44.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:08:44.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:44.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:08:44.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:08:44.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:44.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:08:44.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:08:44.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:44.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:08:44.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:08:44.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:44.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:44.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:44.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:08:44.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:08:44.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:44.442 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:44.442 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:44.446 INFO:tasks.rados.rados.0.smithi001.stdout:197: done (5 left) 2021-01-15T19:08:44.447 INFO:tasks.rados.rados.0.smithi001.stdout:198: done (4 left) 2021-01-15T19:08:44.447 INFO:tasks.rados.rados.0.smithi001.stdout:200: done (3 left) 2021-01-15T19:08:44.447 INFO:tasks.rados.rados.0.smithi001.stdout:201: done (2 left) 2021-01-15T19:08:44.447 INFO:tasks.rados.rados.0.smithi001.stdout:202: copy_from oid 5 from oid 20 current snap is 17 2021-01-15T19:08:44.448 INFO:tasks.rados.rados.0.smithi001.stdout:203: copy_from oid 29 from oid 28 current snap is 17 2021-01-15T19:08:44.448 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 26 current snap is 17 2021-01-15T19:08:44.448 INFO:tasks.rados.rados.0.smithi001.stdout:204: seq_num 103 ranges {2686976=532480,3219456=737280,3956736=466944} 2021-01-15T19:08:44.448 INFO:tasks.rados.rados.0.smithi001.stdout:204: writing smithi00140323-26 from 2686976 to 3219456 tid 1 2021-01-15T19:08:44.449 INFO:tasks.rados.rados.0.smithi001.stdout:204: writing smithi00140323-26 from 3219456 to 3956736 tid 2 2021-01-15T19:08:44.449 INFO:tasks.rados.rados.0.smithi001.stdout:204: writing smithi00140323-26 from 3956736 to 4423680 tid 3 2021-01-15T19:08:44.449 INFO:tasks.rados.rados.0.smithi001.stdout:205: rollback oid 13 current snap is 17 2021-01-15T19:08:44.449 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 13 to 16 2021-01-15T19:08:44.449 INFO:tasks.rados.rados.0.smithi001.stdout:206: copy_from oid 43 from oid 7 current snap is 17 2021-01-15T19:08:44.450 INFO:tasks.rados.rados.0.smithi001.stdout:207: rollback oid 48 current snap is 17 2021-01-15T19:08:44.450 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 48 to 14 2021-01-15T19:08:44.450 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 8 current snap is 17 2021-01-15T19:08:44.450 INFO:tasks.rados.rados.0.smithi001.stdout:208: seq_num 104 ranges {1810432=548864,2359296=622592,2981888=581632} 2021-01-15T19:08:44.450 INFO:tasks.rados.rados.0.smithi001.stdout:208: writing smithi00140323-8 from 1810432 to 2359296 tid 1 2021-01-15T19:08:44.451 INFO:tasks.rados.rados.0.smithi001.stdout:208: writing smithi00140323-8 from 2359296 to 2981888 tid 2 2021-01-15T19:08:44.451 INFO:tasks.rados.rados.0.smithi001.stdout:208: writing smithi00140323-8 from 2981888 to 3563520 tid 3 2021-01-15T19:08:44.451 INFO:tasks.rados.rados.0.smithi001.stdout:206: finishing copy_from to smithi00140323-43 2021-01-15T19:08:44.451 INFO:tasks.rados.rados.0.smithi001.stdout:206: got expected ENOENT (src dne) 2021-01-15T19:08:44.451 INFO:tasks.rados.rados.0.smithi001.stdout:206: finishing copy_from racing read to smithi00140323-43 2021-01-15T19:08:44.452 INFO:tasks.rados.rados.0.smithi001.stdout:205: finishing rollback tid 0 to smithi00140323-13 2021-01-15T19:08:44.452 INFO:tasks.rados.rados.0.smithi001.stdout:202: finishing copy_from to smithi00140323-5 2021-01-15T19:08:44.452 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 8 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:08:44.453 INFO:tasks.rados.rados.0.smithi001.stdout:205: finishing rollback tid 1 to smithi00140323-13 2021-01-15T19:08:44.453 INFO:tasks.rados.rados.0.smithi001.stdout:205: finishing rollback tid 2 to smithi00140323-13 2021-01-15T19:08:44.453 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 52 (ObjNum 98 snap 16 seq_num 98) dirty exists 2021-01-15T19:08:44.453 INFO:tasks.rados.rados.0.smithi001.stdout:202: finishing copy_from racing read to smithi00140323-5 2021-01-15T19:08:44.453 INFO:tasks.rados.rados.0.smithi001.stdout:202: done (8 left) 2021-01-15T19:08:44.453 INFO:tasks.rados.rados.0.smithi001.stdout:205: done (7 left) 2021-01-15T19:08:44.454 INFO:tasks.rados.rados.0.smithi001.stdout:206: done (6 left) 2021-01-15T19:08:44.454 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 37 current snap is 17 2021-01-15T19:08:44.454 INFO:tasks.rados.rados.0.smithi001.stdout:209: seq_num 105 ranges {1564672=565248,2129920=507904,2637824=507904,3145728=188416} 2021-01-15T19:08:44.454 INFO:tasks.rados.rados.0.smithi001.stdout:209: writing smithi00140323-37 from 1564672 to 2129920 tid 1 2021-01-15T19:08:44.454 INFO:tasks.rados.rados.0.smithi001.stdout:209: writing smithi00140323-37 from 2129920 to 2637824 tid 2 2021-01-15T19:08:44.455 INFO:tasks.rados.rados.0.smithi001.stdout:209: writing smithi00140323-37 from 2637824 to 3145728 tid 3 2021-01-15T19:08:44.455 INFO:tasks.rados.rados.0.smithi001.stdout:209: writing smithi00140323-37 from 3145728 to 3334144 tid 4 2021-01-15T19:08:44.455 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 41 current snap is 17 2021-01-15T19:08:44.455 INFO:tasks.rados.rados.0.smithi001.stdout:208: finishing write tid 1 to smithi00140323-8 2021-01-15T19:08:44.456 INFO:tasks.rados.rados.0.smithi001.stdout:208: finishing write tid 2 to smithi00140323-8 2021-01-15T19:08:44.456 INFO:tasks.rados.rados.0.smithi001.stdout:208: finishing write tid 3 to smithi00140323-8 2021-01-15T19:08:44.456 INFO:tasks.rados.rados.0.smithi001.stdout:210: seq_num 106 ranges {679936=581632,1261568=794624,2056192=409600} 2021-01-15T19:08:44.456 INFO:tasks.rados.rados.0.smithi001.stdout:210: writing smithi00140323-41 from 679936 to 1261568 tid 1 2021-01-15T19:08:44.456 INFO:tasks.rados.rados.0.smithi001.stdout:210: writing smithi00140323-41 from 1261568 to 2056192 tid 2 2021-01-15T19:08:44.457 INFO:tasks.rados.rados.0.smithi001.stdout:210: writing smithi00140323-41 from 2056192 to 2465792 tid 3 2021-01-15T19:08:44.457 INFO:tasks.rados.rados.0.smithi001.stdout:211: snap_create 2021-01-15T19:08:44.457 INFO:tasks.rados.rados.0.smithi001.stdout:208: finishing write tid 4 to smithi00140323-8 2021-01-15T19:08:44.457 INFO:tasks.rados.rados.0.smithi001.stdout:208: finishing write tid 5 to smithi00140323-8 2021-01-15T19:08:44.458 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 38 (ObjNum 104 snap 17 seq_num 104) dirty exists 2021-01-15T19:08:44.458 INFO:tasks.rados.rados.0.smithi001.stdout:208: left oid 8 (ObjNum 104 snap 17 seq_num 104) 2021-01-15T19:08:44.458 INFO:tasks.rados.rados.0.smithi001.stdout:207: finishing rollback tid 0 to smithi00140323-48 2021-01-15T19:08:44.458 INFO:tasks.rados.rados.0.smithi001.stdout:207: finishing rollback tid 1 to smithi00140323-48 2021-01-15T19:08:44.458 INFO:tasks.rados.rados.0.smithi001.stdout:203: finishing copy_from to smithi00140323-29 2021-01-15T19:08:44.459 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 23 (ObjNum 80 snap 14 seq_num 80) dirty exists 2021-01-15T19:08:44.459 INFO:tasks.rados.rados.0.smithi001.stdout:199: finishing rollback tid 0 to smithi00140323-40 2021-01-15T19:08:44.459 INFO:tasks.rados.rados.0.smithi001.stdout:199: finishing rollback tid 1 to smithi00140323-40 2021-01-15T19:08:44.459 INFO:tasks.rados.rados.0.smithi001.stdout:207: finishing rollback tid 2 to smithi00140323-48 2021-01-15T19:08:44.459 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 53 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:08:44.460 INFO:tasks.rados.rados.0.smithi001.stdout:199: finishing rollback tid 2 to smithi00140323-40 2021-01-15T19:08:44.460 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 40 v 37 (ObjNum 63 snap 7 seq_num 63) dirty exists 2021-01-15T19:08:44.460 INFO:tasks.rados.rados.0.smithi001.stdout:210: finishing write tid 1 to smithi00140323-41 2021-01-15T19:08:44.460 INFO:tasks.rados.rados.0.smithi001.stdout:203: finishing copy_from racing read to smithi00140323-29 2021-01-15T19:08:44.460 INFO:tasks.rados.rados.0.smithi001.stdout:210: finishing write tid 2 to smithi00140323-41 2021-01-15T19:08:44.461 INFO:tasks.rados.rados.0.smithi001.stdout:210: finishing write tid 3 to smithi00140323-41 2021-01-15T19:08:44.461 INFO:tasks.rados.rados.0.smithi001.stdout:210: finishing write tid 4 to smithi00140323-41 2021-01-15T19:08:44.461 INFO:tasks.rados.rados.0.smithi001.stdout:210: finishing write tid 5 to smithi00140323-41 2021-01-15T19:08:44.461 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 41 v 13 (ObjNum 106 snap 17 seq_num 106) dirty exists 2021-01-15T19:08:44.462 INFO:tasks.rados.rados.0.smithi001.stdout:210: left oid 41 (ObjNum 106 snap 17 seq_num 106) 2021-01-15T19:08:44.462 INFO:tasks.rados.rados.0.smithi001.stdout:188: expect (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:08:44.462 INFO:tasks.rados.rados.0.smithi001.stdout:209: finishing write tid 1 to smithi00140323-37 2021-01-15T19:08:44.462 INFO:tasks.rados.rados.0.smithi001.stdout:209: finishing write tid 2 to smithi00140323-37 2021-01-15T19:08:44.463 INFO:tasks.rados.rados.0.smithi001.stdout:204: finishing write tid 1 to smithi00140323-26 2021-01-15T19:08:44.463 INFO:tasks.rados.rados.0.smithi001.stdout:209: finishing write tid 3 to smithi00140323-37 2021-01-15T19:08:44.463 INFO:tasks.rados.rados.0.smithi001.stdout:209: finishing write tid 4 to smithi00140323-37 2021-01-15T19:08:44.463 INFO:tasks.rados.rados.0.smithi001.stdout:204: finishing write tid 2 to smithi00140323-26 2021-01-15T19:08:44.463 INFO:tasks.rados.rados.0.smithi001.stdout:209: finishing write tid 5 to smithi00140323-37 2021-01-15T19:08:44.464 INFO:tasks.rados.rados.0.smithi001.stdout:204: finishing write tid 3 to smithi00140323-26 2021-01-15T19:08:44.464 INFO:tasks.rados.rados.0.smithi001.stdout:204: finishing write tid 4 to smithi00140323-26 2021-01-15T19:08:44.464 INFO:tasks.rados.rados.0.smithi001.stdout:209: finishing write tid 6 to smithi00140323-37 2021-01-15T19:08:44.464 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 30 (ObjNum 105 snap 17 seq_num 105) dirty exists 2021-01-15T19:08:44.464 INFO:tasks.rados.rados.0.smithi001.stdout:209: left oid 37 (ObjNum 105 snap 17 seq_num 105) 2021-01-15T19:08:44.464 INFO:tasks.rados.rados.0.smithi001.stdout:204: finishing write tid 5 to smithi00140323-26 2021-01-15T19:08:44.465 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 40 (ObjNum 103 snap 17 seq_num 103) dirty exists 2021-01-15T19:08:44.465 INFO:tasks.rados.rados.0.smithi001.stdout:204: left oid 26 (ObjNum 103 snap 17 seq_num 103) 2021-01-15T19:08:44.466 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:44.442+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:44.466 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:44.560 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:44.545+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:44.607 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:44.614 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:44.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:44.657 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:44.655+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:44.749 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:44.747+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:44.761 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:44.789 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:44.791 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:44.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:44.849 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:44.848+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:44.934 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:44.952 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:44.948+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:44.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:44.976 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:44.976 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:44.977 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:44.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:44.998 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:45.018 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:45.040 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:45.051 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:45.049+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:45.062 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:45.123 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:45.124 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:45.133 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:45.152 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:45.151+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:45.181 INFO:tasks.rados.rados.0.smithi001.stdout:188: done (8 left) 2021-01-15T19:08:45.181 INFO:tasks.rados.rados.0.smithi001.stdout:199: done (7 left) 2021-01-15T19:08:45.182 INFO:tasks.rados.rados.0.smithi001.stdout:203: done (6 left) 2021-01-15T19:08:45.182 INFO:tasks.rados.rados.0.smithi001.stdout:204: done (5 left) 2021-01-15T19:08:45.182 INFO:tasks.rados.rados.0.smithi001.stdout:207: done (4 left) 2021-01-15T19:08:45.182 INFO:tasks.rados.rados.0.smithi001.stdout:208: done (3 left) 2021-01-15T19:08:45.183 INFO:tasks.rados.rados.0.smithi001.stdout:209: done (2 left) 2021-01-15T19:08:45.183 INFO:tasks.rados.rados.0.smithi001.stdout:210: done (1 left) 2021-01-15T19:08:45.184 INFO:tasks.rados.rados.0.smithi001.stdout:211: done (0 left) 2021-01-15T19:08:45.184 INFO:tasks.rados.rados.0.smithi001.stdout:212: snap_create 2021-01-15T19:08:45.194 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 1 pg_num to 11 2021-01-15T19:08:45.252 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:45.251+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:45.281 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:45.281 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:45.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:45.374 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:45.373+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:45.455 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:45.453+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:45.462 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:45.463 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:45.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:45.517 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:08:45.560 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:45.554+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:45.653 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:45.654 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:45.658 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:45.656+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:45.663 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:45.759 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:45.757+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:45.796 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:45.797 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:45.806 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:45.864 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:45.860+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:45.938 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:45.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:45.962 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:45.961+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:46.062 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:46.061+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:46.121 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:46.121 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:46.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:46.176 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:46.174+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:46.182 INFO:tasks.rados.rados.0.smithi001.stdout:212: done (0 left) 2021-01-15T19:08:46.182 INFO:tasks.rados.rados.0.smithi001.stdout:213: snap_create 2021-01-15T19:08:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:08:46.264 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:46.263+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:46.287 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:46.287 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:46.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:46.379 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:46.378+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:46.430 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:46.431 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:46.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:46.481 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:46.479+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:46.581 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:46.581 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:46.582 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:46.580+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:46.595 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:46.684 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:46.683+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:46.764 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:46.764 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:46.773 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:46.815 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:46.813+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:46.893 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:46.892+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:46.939 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:46.939 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:46.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:46.995 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:46.994+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:47.077 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:47.118 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:47.119 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:47.123 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:47.121+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:47.124 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:47.183 INFO:tasks.rados.rados.0.smithi001.stdout:213: done (0 left) 2021-01-15T19:08:47.184 INFO:tasks.rados.rados.0.smithi001.stdout:214: snap_create 2021-01-15T19:08:47.198 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:47.196+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:47.246 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:47.265 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:47.265 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:47.266 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:47.267 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:47.286 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:47.308 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:47.328 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:47.329 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:47.313+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:47.350 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:47.371 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:47.394 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:47.395 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.395 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.395 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.395 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.396 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.396 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.396 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.396 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:47.399 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:47.399 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.4s1 36/21 ECSubWrite(tid=44, reqid=client.4418.0:240, at_version=36'14, trim_to=0'0, roll_forward_to=35'12))", 2021-01-15T19:08:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285963+0000", 2021-01-15T19:08:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.109141383000001, 2021-01-15T19:08:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079052319999999999, 2021-01-15T19:08:47.402 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285963+0000", 2021-01-15T19:08:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285963+0000", 2021-01-15T19:08:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000204279 2021-01-15T19:08:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286168+0000", 2021-01-15T19:08:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3572000000000001e-05 2021-01-15T19:08:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286191+0000", 2021-01-15T19:08:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7017e-05 2021-01-15T19:08:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286208+0000", 2021-01-15T19:08:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.088e-06 2021-01-15T19:08:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286214+0000", 2021-01-15T19:08:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044245899999999997 2021-01-15T19:08:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286657+0000", 2021-01-15T19:08:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2899e-05 2021-01-15T19:08:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286670+0000", 2021-01-15T19:08:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071481629999999999 2021-01-15T19:08:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.293818+0000", 2021-01-15T19:08:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754999999999998e-05 2021-01-15T19:08:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.293869+0000", 2021-01-15T19:08:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079052319999999999 2021-01-15T19:08:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:08:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.109070224, 2021-01-15T19:08:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:08:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:08:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:47.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:08:47.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:08:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:08:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:08:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:08:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:08:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:08:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:08:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:08:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:08:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:08:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:08:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:08:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:08:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:08:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:08:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:08:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:08:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:08:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:08:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:08:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:08:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:08:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:08:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:08:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.177307458000001, 2021-01-15T19:08:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:08:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:08:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:08:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:08:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:08:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:08:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:08:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:08:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:08:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:08:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:08:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:08:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:08:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:08:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:08:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:08:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:08:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:08:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:08:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:08:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:08:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:08:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:08:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:08:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:08:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:08:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:08:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.175019417000001, 2021-01-15T19:08:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:08:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:08:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:08:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:08:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:08:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:08:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:08:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:08:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:08:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:08:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:08:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:08:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:08:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:08:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:08:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:08:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:08:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:08:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:08:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:08:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:08:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:08:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:08:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.172429791999999, 2021-01-15T19:08:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:08:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:08:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:47.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:08:47.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:08:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:08:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:08:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:08:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:08:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:08:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:08:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:08:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:08:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:08:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:08:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:08:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:08:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:08:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:08:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:08:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:08:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:08:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:08:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:08:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:335 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (56) in=7b] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.171622348, 2021-01-15T19:08:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086755649999999997, 2021-01-15T19:08:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2021-01-15T19:08:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2554e-05 2021-01-15T19:08:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223515+0000", 2021-01-15T19:08:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.776e-06 2021-01-15T19:08:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223522+0000", 2021-01-15T19:08:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2351000000000001e-05 2021-01-15T19:08:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223534+0000", 2021-01-15T19:08:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4670000000000001e-06 2021-01-15T19:08:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223541+0000", 2021-01-15T19:08:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028742450000000001 2021-01-15T19:08:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226415+0000", 2021-01-15T19:08:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2873000000000005e-05 2021-01-15T19:08:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226488+0000", 2021-01-15T19:08:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000392001 2021-01-15T19:08:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226880+0000", 2021-01-15T19:08:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061445900000000003 2021-01-15T19:08:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227494+0000", 2021-01-15T19:08:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045987420000000003 2021-01-15T19:08:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232093+0000", 2021-01-15T19:08:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5097000000000004e-05 2021-01-15T19:08:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232158+0000", 2021-01-15T19:08:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086755649999999997 2021-01-15T19:08:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.171521513999998, 2021-01-15T19:08:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:08:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:47.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:08:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:08:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:08:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:08:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:08:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:08:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:08:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:08:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:08:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:08:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:08:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:08:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:08:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:08:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:08:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:08:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:08:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:08:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:08:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:08:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.202837606999999, 2021-01-15T19:08:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:08:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:08:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:08:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:08:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:08:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:08:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:08:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:08:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:08:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:08:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:08:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:08:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:08:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:08:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:08:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:08:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:08:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:08:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:08:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:08:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:08:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:08:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:08:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:08:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:08:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:08:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:08:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.20268787, 2021-01-15T19:08:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:08:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:08:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:08:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:08:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:08:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:08:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:08:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:08:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:08:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:08:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:08:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:08:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:08:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:08:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:08:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:08:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:08:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:08:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:08:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:08:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:08:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:08:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:08:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:08:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:08:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:08:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:08:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:08:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:08:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.202535972, 2021-01-15T19:08:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:08:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:08:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:08:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:08:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:08:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:08:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:08:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:08:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:08:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:08:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:08:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:08:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:08:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:08:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:08:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:08:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:08:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:08:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:08:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:08:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:08:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:08:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:08:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:08:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:08:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:08:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:08:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.20241066, 2021-01-15T19:08:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:08:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:08:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:08:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:08:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:08:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:08:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:08:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:08:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:08:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:08:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:08:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:08:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:08:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:08:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:08:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:08:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:08:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:08:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:08:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:08:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:08:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:08:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:08:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:08:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:08:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:08:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:08:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:08:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:08:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:08:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.181578085, 2021-01-15T19:08:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:08:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:08:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:08:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:08:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:08:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:08:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:08:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:08:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:08:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:08:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:08:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:08:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:08:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:08:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:08:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:08:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:08:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:08:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:08:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:08:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:08:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:08:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:08:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:08:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.175284753, 2021-01-15T19:08:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:08:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:08:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:08:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:08:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:08:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:08:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:08:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:08:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:08:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:08:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:08:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:08:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:08:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:08:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:08:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:08:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:08:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:08:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:08:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:08:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:08:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:08:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.2132252399999999, 2021-01-15T19:08:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:08:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:08:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:08:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:08:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:08:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:08:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:08:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:08:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:08:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:08:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:08:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:08:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:08:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:08:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:08:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:08:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:08:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:08:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:08:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:08:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:08:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:08:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:08:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:08:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:08:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:08:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:08:47.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.1508532599999999, 2021-01-15T19:08:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:08:47.672 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:08:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:08:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:08:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:08:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:08:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:08:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:08:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:08:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:08:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:08:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:08:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:08:47.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:08:47.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:08:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:08:47.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:08:47.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:08:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:08:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:08:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:08:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:08:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:08:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.689 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:47.691 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.1475507589999996, 2021-01-15T19:08:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:08:47.692 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:08:47.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:47.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:08:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:08:47.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:08:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:08:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:08:47.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:08:47.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:08:47.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:08:47.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:08:47.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:08:47.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:08:47.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:08:47.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:08:47.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:08:47.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:08:47.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:08:47.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:08:47.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:08:47.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:08:47.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:08:47.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:08:47.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.732 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.732 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.733 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:47.733 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:47.733 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.146228893, 2021-01-15T19:08:47.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:08:47.733 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.734 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.734 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:08:47.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.735 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:47.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:47.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:08:47.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:08:47.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:08:47.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:08:47.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:08:47.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:08:47.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:08:47.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:08:47.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:08:47.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:08:47.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:08:47.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:08:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:08:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:08:47.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:08:47.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:08:47.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:08:47.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:08:47.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:08:47.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:08:47.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:08:47.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.749 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:47.749 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:47.749 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.1458586400000002, 2021-01-15T19:08:47.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:08:47.749 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.750 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:47.750 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.750 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.750 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.750 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:08:47.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.751 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:47.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:47.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:08:47.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:08:47.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:08:47.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:08:47.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:08:47.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:08:47.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:08:47.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:08:47.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:08:47.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:08:47.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:08:47.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:08:47.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:08:47.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:47.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:08:47.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:08:47.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:47.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:08:47.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:08:47.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:47.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:08:47.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:08:47.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:08:47.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:08:47.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.764 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.765 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:08:47.765 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:47.765 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.1455242470000009, 2021-01-15T19:08:47.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:08:47.766 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:47.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:47.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:47.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:47.767 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:08:47.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.767 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:47.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:47.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:08:47.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:08:47.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:08:47.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:08:47.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:08:47.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:08:47.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:08:47.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:08:47.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:08:47.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:08:47.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:08:47.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:47.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:08:47.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:08:47.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:08:47.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:08:47.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:08:47.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:08:47.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:08:47.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:08:47.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.779 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.780 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.3.13:2 3.0s0 3:0c0290e4:::smithi00140323-22:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:47.780 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:47.780 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.1441746829999992, 2021-01-15T19:08:47.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082549809999999998, 2021-01-15T19:08:47.781 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:47.781 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:47.781 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:47.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:47.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:47.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:47.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:47.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:47.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9704999999999997e-05 2021-01-15T19:08:47.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:47.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251010+0000", 2021-01-15T19:08:47.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5044e-05 2021-01-15T19:08:47.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:47.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251025+0000", 2021-01-15T19:08:47.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.844e-05 2021-01-15T19:08:47.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:47.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251043+0000", 2021-01-15T19:08:47.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1319999999999996e-06 2021-01-15T19:08:47.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:47.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251050+0000", 2021-01-15T19:08:47.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029025309999999999 2021-01-15T19:08:47.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:47.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253953+0000", 2021-01-15T19:08:47.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010406199999999999 2021-01-15T19:08:47.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254057+0000", 2021-01-15T19:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005128067 2021-01-15T19:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:47.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:47.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:47.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259185+0000", 2021-01-15T19:08:47.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082549809999999998 2021-01-15T19:08:47.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.792 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:47.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:47.793 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:47.794 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:47.793+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:47.795 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:47.895 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:47.894+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:47.963 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:47.963 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:47.972 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:47.996 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:47.995+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:48.098 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:48.096+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:48.137 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:48.138 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:48.138 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:48.138 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:48.139 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:48.147 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:48.184 INFO:tasks.rados.rados.0.smithi001.stdout:214: done (0 left) 2021-01-15T19:08:48.185 INFO:tasks.rados.rados.0.smithi001.stdout:215: snap_create 2021-01-15T19:08:48.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:08:48.233 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:48.231+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:48.300 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:48.298+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:48.319 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.060644138999997, 2021-01-15T19:08:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.057917498999998, 2021-01-15T19:08:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.056993290000001, 2021-01-15T19:08:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:48.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:48.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.056769326999998, 2021-01-15T19:08:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:48.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:48.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:48.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:48.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:48.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:48.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:48.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:48.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:48.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:48.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:48.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.417 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:48.417 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:48.418 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.032410294000002, 2021-01-15T19:08:48.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:48.422 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.423 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.423 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.423 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:48.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.424 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:48.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:48.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:48.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:48.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:48.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:48.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:48.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:48.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:48.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:48.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:48.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:48.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:48.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:48.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:48.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:48.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:48.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:48.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:48.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:48.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:48.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:48.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:48.439 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.439 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.439 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.441 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:48.441 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:48.442 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.030231090000001, 2021-01-15T19:08:48.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:48.442 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.443 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.444 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:48.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.445 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:48.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:48.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:48.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:48.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:48.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:48.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:48.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:48.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:48.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:48.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:48.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:48.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:48.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:48.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:48.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:48.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:48.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:48.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:48.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:48.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:48.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:48.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:48.468 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.468 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.470 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:48.470 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:48.470 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.029476725999999, 2021-01-15T19:08:48.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:48.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.472 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:48.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.473 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:48.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:48.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:48.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:48.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:48.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:48.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:48.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:48.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:48.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:48.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:48.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:48.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:48.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:48.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:48.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:48.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:48.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:48.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:48.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:48.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:48.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:48.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:48.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.500 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.501 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:48.501 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:48.501 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.02935656, 2021-01-15T19:08:48.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:48.502 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.502 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.502 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.503 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:48.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.504 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:48.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:48.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:48.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:48.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:48.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:48.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:48.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:48.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:48.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:48.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:48.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:48.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:48.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:48.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:48.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:48.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:48.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:48.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:48.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:48.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:48.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:48.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.518 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.519 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:318 3.1s0 3:8b30e322:::smithi00140323-24:head [copy-from ver 8] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:48.519 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:48.519 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.120155962999998, 2021-01-15T19:08:48.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729, 2021-01-15T19:08:48.519 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.520 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.521 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 318 2021-01-15T19:08:48.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.521 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:48.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:48.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2021-01-15T19:08:48.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196102+0000", 2021-01-15T19:08:48.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6286e-05 2021-01-15T19:08:48.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196119+0000", 2021-01-15T19:08:48.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0645e-05 2021-01-15T19:08:48.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196139+0000", 2021-01-15T19:08:48.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576000000000001e-05 2021-01-15T19:08:48.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196151+0000", 2021-01-15T19:08:48.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017478200000000001 2021-01-15T19:08:48.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196326+0000", 2021-01-15T19:08:48.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019917200000000001 2021-01-15T19:08:48.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196525+0000", 2021-01-15T19:08:48.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099001019999999992 2021-01-15T19:08:48.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.206425+0000", 2021-01-15T19:08:48.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043897240000000002 2021-01-15T19:08:48.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210815+0000", 2021-01-15T19:08:48.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023466619999999998 2021-01-15T19:08:48.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213161+0000", 2021-01-15T19:08:48.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108811 2021-01-15T19:08:48.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213270+0000", 2021-01-15T19:08:48.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729 2021-01-15T19:08:48.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.561 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.562 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:319 3.1s0 3:8b30e322:::smithi00140323-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:48.562 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:48.562 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.119993045000001, 2021-01-15T19:08:48.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001, 2021-01-15T19:08:48.562 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.563 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.563 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 319 2021-01-15T19:08:48.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.564 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:48.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:48.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6667e-05 2021-01-15T19:08:48.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196236+0000", 2021-01-15T19:08:48.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:48.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196243+0000", 2021-01-15T19:08:48.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4992e-05 2021-01-15T19:08:48.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196258+0000", 2021-01-15T19:08:48.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6110000000000001e-06 2021-01-15T19:08:48.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196264+0000", 2021-01-15T19:08:48.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046740399999999999 2021-01-15T19:08:48.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196731+0000", 2021-01-15T19:08:48.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1105000000000004e-05 2021-01-15T19:08:48.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:48.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196812+0000", 2021-01-15T19:08:48.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011315451000000001 2021-01-15T19:08:48.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208128+0000", 2021-01-15T19:08:48.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010873 2021-01-15T19:08:48.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:48.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208237+0000", 2021-01-15T19:08:48.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058576419999999997 2021-01-15T19:08:48.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214094+0000", 2021-01-15T19:08:48.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119562 2021-01-15T19:08:48.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214214+0000", 2021-01-15T19:08:48.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012588000000000001 2021-01-15T19:08:48.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214340+0000", 2021-01-15T19:08:48.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001 2021-01-15T19:08:48.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:494 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:48.585 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:48.585 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1329796060000001, 2021-01-15T19:08:48.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1107716759999999, 2021-01-15T19:08:48.585 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.586 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.592 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 494 2021-01-15T19:08:48.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.592 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:48.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:48.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.092e-05 2021-01-15T19:08:48.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183264+0000", 2021-01-15T19:08:48.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2300000000000008e-06 2021-01-15T19:08:48.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183272+0000", 2021-01-15T19:08:48.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6365e-05 2021-01-15T19:08:48.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183288+0000", 2021-01-15T19:08:48.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3640000000000001e-06 2021-01-15T19:08:48.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183295+0000", 2021-01-15T19:08:48.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020715730000000002 2021-01-15T19:08:48.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185367+0000", 2021-01-15T19:08:48.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.627e-06 2021-01-15T19:08:48.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:48.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185370+0000", 2021-01-15T19:08:48.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0069268849999999 2021-01-15T19:08:48.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.192297+0000", 2021-01-15T19:08:48.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036647600000000002 2021-01-15T19:08:48.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.192663+0000", 2021-01-15T19:08:48.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.095695180000000005 2021-01-15T19:08:48.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.288358+0000", 2021-01-15T19:08:48.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112618 2021-01-15T19:08:48.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.288471+0000", 2021-01-15T19:08:48.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055134379999999998 2021-01-15T19:08:48.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.293984+0000", 2021-01-15T19:08:48.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1107716759999999 2021-01-15T19:08:48.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.606 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.607 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:495 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:48.607 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:48.607 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1312958980000003, 2021-01-15T19:08:48.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111341001, 2021-01-15T19:08:48.608 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.608 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.609 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 495 2021-01-15T19:08:48.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.609 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:48.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:48.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4907999999999997e-05 2021-01-15T19:08:48.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184951+0000", 2021-01-15T19:08:48.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9130000000000003e-06 2021-01-15T19:08:48.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184959+0000", 2021-01-15T19:08:48.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4218e-05 2021-01-15T19:08:48.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184973+0000", 2021-01-15T19:08:48.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.212e-06 2021-01-15T19:08:48.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184980+0000", 2021-01-15T19:08:48.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000418804 2021-01-15T19:08:48.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185398+0000", 2021-01-15T19:08:48.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2021-01-15T19:08:48.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:48.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185399+0000", 2021-01-15T19:08:48.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.007701593 2021-01-15T19:08:48.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193101+0000", 2021-01-15T19:08:48.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010404 2021-01-15T19:08:48.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193205+0000", 2021-01-15T19:08:48.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099598102999999993 2021-01-15T19:08:48.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.292803+0000", 2021-01-15T19:08:48.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122609 2021-01-15T19:08:48.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.292925+0000", 2021-01-15T19:08:48.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033118269999999998 2021-01-15T19:08:48.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.296237+0000", 2021-01-15T19:08:48.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111341001 2021-01-15T19:08:48.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.625 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.626 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:496 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:48.626 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:48.626 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1311714410000002, 2021-01-15T19:08:48.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1128853430000001, 2021-01-15T19:08:48.627 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.627 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 496 2021-01-15T19:08:48.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:48.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:48.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5720000000000002e-05 2021-01-15T19:08:48.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185076+0000", 2021-01-15T19:08:48.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9819999999999997e-06 2021-01-15T19:08:48.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185085+0000", 2021-01-15T19:08:48.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3526000000000001e-05 2021-01-15T19:08:48.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185099+0000", 2021-01-15T19:08:48.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999998e-06 2021-01-15T19:08:48.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185105+0000", 2021-01-15T19:08:48.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036220000000000002 2021-01-15T19:08:48.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185467+0000", 2021-01-15T19:08:48.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:08:48.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:48.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185469+0000", 2021-01-15T19:08:48.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.007797008 2021-01-15T19:08:48.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193266+0000", 2021-01-15T19:08:48.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102037 2021-01-15T19:08:48.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193368+0000", 2021-01-15T19:08:48.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10068078 2021-01-15T19:08:48.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.294049+0000", 2021-01-15T19:08:48.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010173 2021-01-15T19:08:48.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.294151+0000", 2021-01-15T19:08:48.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037554530000000002 2021-01-15T19:08:48.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.297906+0000", 2021-01-15T19:08:48.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1128853430000001 2021-01-15T19:08:48.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.649 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.650 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:507 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:08:48.650 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:48.650 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0837836630000002, 2021-01-15T19:08:48.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013129708, 2021-01-15T19:08:48.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.651 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.653 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2021-01-15T19:08:48.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.653 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:48.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:48.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7067000000000003e-05 2021-01-15T19:08:48.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232446+0000", 2021-01-15T19:08:48.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7929999999999992e-06 2021-01-15T19:08:48.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232455+0000", 2021-01-15T19:08:48.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1325000000000001e-05 2021-01-15T19:08:48.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232467+0000", 2021-01-15T19:08:48.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6519999999999997e-06 2021-01-15T19:08:48.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232471+0000", 2021-01-15T19:08:48.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006329187 2021-01-15T19:08:48.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.238801+0000", 2021-01-15T19:08:48.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010693569999999999 2021-01-15T19:08:48.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:48.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.239870+0000", 2021-01-15T19:08:48.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011900960000000001 2021-01-15T19:08:48.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.241060+0000", 2021-01-15T19:08:48.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012403300000000001 2021-01-15T19:08:48.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.241184+0000", 2021-01-15T19:08:48.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043541980000000001 2021-01-15T19:08:48.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.245538+0000", 2021-01-15T19:08:48.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013129708 2021-01-15T19:08:48.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.678 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:572 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~565248] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:48.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:48.681 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0652136480000003, 2021-01-15T19:08:48.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.058327720999999999, 2021-01-15T19:08:48.682 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.691 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.692 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 572 2021-01-15T19:08:48.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:48.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:48.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010807060000000001 2021-01-15T19:08:48.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252059+0000", 2021-01-15T19:08:48.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.392e-06 2021-01-15T19:08:48.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252066+0000", 2021-01-15T19:08:48.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1586999999999999e-05 2021-01-15T19:08:48.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252077+0000", 2021-01-15T19:08:48.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8189999999999999e-06 2021-01-15T19:08:48.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252082+0000", 2021-01-15T19:08:48.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4982000000000003e-05 2021-01-15T19:08:48.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252137+0000", 2021-01-15T19:08:48.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000471743 2021-01-15T19:08:48.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252609+0000", 2021-01-15T19:08:48.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045366254000000002 2021-01-15T19:08:48.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.297975+0000", 2021-01-15T19:08:48.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106475 2021-01-15T19:08:48.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.298082+0000", 2021-01-15T19:08:48.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086370999999999996 2021-01-15T19:08:48.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.298945+0000", 2021-01-15T19:08:48.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068992059999999997 2021-01-15T19:08:48.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305844+0000", 2021-01-15T19:08:48.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032253609999999999 2021-01-15T19:08:48.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309070+0000", 2021-01-15T19:08:48.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236486 2021-01-15T19:08:48.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309306+0000", 2021-01-15T19:08:48.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.058327720999999999 2021-01-15T19:08:48.712 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.712 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.713 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.713 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:573 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~507904] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:48.714 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:48.714 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0614106870000004, 2021-01-15T19:08:48.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.057854161000000001, 2021-01-15T19:08:48.715 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.715 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.717 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 573 2021-01-15T19:08:48.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.718 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:48.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:48.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050420300000000005 2021-01-15T19:08:48.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255286+0000", 2021-01-15T19:08:48.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:48.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255294+0000", 2021-01-15T19:08:48.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6209999999999999e-05 2021-01-15T19:08:48.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255310+0000", 2021-01-15T19:08:48.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2570000000000001e-06 2021-01-15T19:08:48.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255317+0000", 2021-01-15T19:08:48.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5633000000000002e-05 2021-01-15T19:08:48.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255363+0000", 2021-01-15T19:08:48.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7139999999999997e-05 2021-01-15T19:08:48.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255440+0000", 2021-01-15T19:08:48.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045807546999999997 2021-01-15T19:08:48.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.301248+0000", 2021-01-15T19:08:48.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4977999999999998e-05 2021-01-15T19:08:48.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.301343+0000", 2021-01-15T19:08:48.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066354300000000005 2021-01-15T19:08:48.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.302006+0000", 2021-01-15T19:08:48.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056809900000000003 2021-01-15T19:08:48.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307687+0000", 2021-01-15T19:08:48.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045007709999999998 2021-01-15T19:08:48.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312188+0000", 2021-01-15T19:08:48.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044770900000000002 2021-01-15T19:08:48.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312636+0000", 2021-01-15T19:08:48.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.057854161000000001 2021-01-15T19:08:48.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:574 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~507904] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:48.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:48.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0576296420000002, 2021-01-15T19:08:48.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055246712000000003, 2021-01-15T19:08:48.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.744 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 574 2021-01-15T19:08:48.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:48.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:48.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000591451 2021-01-15T19:08:48.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259154+0000", 2021-01-15T19:08:48.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9889999999999998e-06 2021-01-15T19:08:48.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259163+0000", 2021-01-15T19:08:48.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4970999999999999e-05 2021-01-15T19:08:48.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259178+0000", 2021-01-15T19:08:48.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.798e-06 2021-01-15T19:08:48.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259185+0000", 2021-01-15T19:08:48.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022518479999999999 2021-01-15T19:08:48.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261437+0000", 2021-01-15T19:08:48.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107135 2021-01-15T19:08:48.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261544+0000", 2021-01-15T19:08:48.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041508633000000003 2021-01-15T19:08:48.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.303052+0000", 2021-01-15T19:08:48.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136876 2021-01-15T19:08:48.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.303189+0000", 2021-01-15T19:08:48.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000916328 2021-01-15T19:08:48.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.304106+0000", 2021-01-15T19:08:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052131690000000001 2021-01-15T19:08:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309319+0000", 2021-01-15T19:08:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041306750000000003 2021-01-15T19:08:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313449+0000", 2021-01-15T19:08:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035983899999999998 2021-01-15T19:08:48.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313809+0000", 2021-01-15T19:08:48.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055246712000000003 2021-01-15T19:08:48.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.759 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:575 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~188416] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:48.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:48.760 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0560633849999999, 2021-01-15T19:08:48.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054232529000000002, 2021-01-15T19:08:48.761 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.761 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.761 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.761 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.761 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.761 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 575 2021-01-15T19:08:48.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.762 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:48.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:48.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000227791 2021-01-15T19:08:48.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260357+0000", 2021-01-15T19:08:48.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2290000000000003e-06 2021-01-15T19:08:48.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260365+0000", 2021-01-15T19:08:48.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6650000000000002e-05 2021-01-15T19:08:48.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260382+0000", 2021-01-15T19:08:48.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1879999999999996e-06 2021-01-15T19:08:48.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260389+0000", 2021-01-15T19:08:48.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001199133 2021-01-15T19:08:48.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261588+0000", 2021-01-15T19:08:48.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4250999999999999e-05 2021-01-15T19:08:48.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261642+0000", 2021-01-15T19:08:48.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044223854999999999 2021-01-15T19:08:48.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305866+0000", 2021-01-15T19:08:48.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8614000000000001e-05 2021-01-15T19:08:48.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305965+0000", 2021-01-15T19:08:48.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060702 2021-01-15T19:08:48.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.306572+0000", 2021-01-15T19:08:48.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006392488 2021-01-15T19:08:48.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312964+0000", 2021-01-15T19:08:48.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010513219999999999 2021-01-15T19:08:48.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314015+0000", 2021-01-15T19:08:48.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034598799999999998 2021-01-15T19:08:48.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314361+0000", 2021-01-15T19:08:48.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054232529000000002 2021-01-15T19:08:48.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.778 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:576 3.12s0 3:4e4266df:::smithi00140323-37:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:48.778 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:48.778 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0557589529999998, 2021-01-15T19:08:48.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054314145000000001, 2021-01-15T19:08:48.778 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.779 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:48.779 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.779 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.779 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.780 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 576 2021-01-15T19:08:48.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.780 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:48.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:48.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5556000000000002e-05 2021-01-15T19:08:48.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260479+0000", 2021-01-15T19:08:48.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0480000000000004e-06 2021-01-15T19:08:48.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260485+0000", 2021-01-15T19:08:48.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3729999999999993e-06 2021-01-15T19:08:48.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260493+0000", 2021-01-15T19:08:48.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9958000000000001e-05 2021-01-15T19:08:48.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260513+0000", 2021-01-15T19:08:48.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00117347 2021-01-15T19:08:48.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261687+0000", 2021-01-15T19:08:48.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5226000000000003e-05 2021-01-15T19:08:48.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261742+0000", 2021-01-15T19:08:48.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045966993999999997 2021-01-15T19:08:48.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307709+0000", 2021-01-15T19:08:48.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3752999999999995e-05 2021-01-15T19:08:48.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307803+0000", 2021-01-15T19:08:48.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057255800000000003 2021-01-15T19:08:48.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:48.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308375+0000", 2021-01-15T19:08:48.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046299920000000003 2021-01-15T19:08:48.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:48.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313005+0000", 2021-01-15T19:08:48.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001483086 2021-01-15T19:08:48.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:48.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314488+0000", 2021-01-15T19:08:48.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025913100000000001 2021-01-15T19:08:48.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314747+0000", 2021-01-15T19:08:48.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054314145000000001 2021-01-15T19:08:48.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.794 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.795 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:577 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:08:48.795 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0556380169999997, 2021-01-15T19:08:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056902372999999999, 2021-01-15T19:08:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:48.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:48.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:48.797 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 577 2021-01-15T19:08:48.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:48.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:48.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:48.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:48.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0988000000000001e-05 2021-01-15T19:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:48.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260595+0000", 2021-01-15T19:08:48.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:48.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:48.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260602+0000", 2021-01-15T19:08:48.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2503e-05 2021-01-15T19:08:48.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:48.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260614+0000", 2021-01-15T19:08:48.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7730000000000002e-06 2021-01-15T19:08:48.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:48.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260621+0000", 2021-01-15T19:08:48.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011648749999999999 2021-01-15T19:08:48.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261786+0000", 2021-01-15T19:08:48.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111148 2021-01-15T19:08:48.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:48.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261897+0000", 2021-01-15T19:08:48.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046774885000000002 2021-01-15T19:08:48.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308672+0000", 2021-01-15T19:08:48.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014140499999999999 2021-01-15T19:08:48.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:48.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308813+0000", 2021-01-15T19:08:48.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063629660000000003 2021-01-15T19:08:48.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:48.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.315176+0000", 2021-01-15T19:08:48.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014743699999999999 2021-01-15T19:08:48.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:48.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.315324+0000", 2021-01-15T19:08:48.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021328549999999999 2021-01-15T19:08:48.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:48.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:48.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:48.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.317457+0000", 2021-01-15T19:08:48.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056902372999999999 2021-01-15T19:08:48.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:48.817 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:48.817 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:48.818 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:48.817+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:48.819 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:48.919 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:48.918+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:48.984 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:48.985 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:48.994 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:49.020 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:49.019+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:49.119 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:49.119 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:49.119 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:49.119 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:49.120 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:49.122 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:49.121+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:49.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:49.191 INFO:tasks.rados.rados.0.smithi001.stdout:215: done (0 left) 2021-01-15T19:08:49.191 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 49 current snap is 22 2021-01-15T19:08:49.191 INFO:tasks.rados.rados.0.smithi001.stdout:216: seq_num 107 ranges {811008=598016,1409024=729088,2138112=466944,2605056=8192} 2021-01-15T19:08:49.199 INFO:tasks.rados.rados.0.smithi001.stdout:216: writing smithi00140323-49 from 811008 to 1409024 tid 1 2021-01-15T19:08:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:08:49.203 INFO:tasks.rados.rados.0.smithi001.stdout:216: writing smithi00140323-49 from 1409024 to 2138112 tid 2 2021-01-15T19:08:49.206 INFO:tasks.rados.rados.0.smithi001.stdout:216: writing smithi00140323-49 from 2138112 to 2605056 tid 3 2021-01-15T19:08:49.206 INFO:tasks.rados.rados.0.smithi001.stdout:216: writing smithi00140323-49 from 2605056 to 2613248 tid 4 2021-01-15T19:08:49.206 INFO:tasks.rados.rados.0.smithi001.stdout:217: read oid 21 snap -1 2021-01-15T19:08:49.207 INFO:tasks.rados.rados.0.smithi001.stdout:217: expect (ObjNum 91 snap 16 seq_num 91) 2021-01-15T19:08:49.207 INFO:tasks.rados.rados.0.smithi001.stdout:218: delete oid 29 current snap is 22 2021-01-15T19:08:49.211 INFO:tasks.rados.rados.0.smithi001.stdout:216: finishing write tid 1 to smithi00140323-49 2021-01-15T19:08:49.215 INFO:tasks.rados.rados.0.smithi001.stdout:218: done (2 left) 2021-01-15T19:08:49.215 INFO:tasks.rados.rados.0.smithi001.stdout:219: snap_create 2021-01-15T19:08:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:216: finishing write tid 2 to smithi00140323-49 2021-01-15T19:08:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:216: finishing write tid 3 to smithi00140323-49 2021-01-15T19:08:49.217 INFO:tasks.rados.rados.0.smithi001.stdout:216: finishing write tid 4 to smithi00140323-49 2021-01-15T19:08:49.217 INFO:tasks.rados.rados.0.smithi001.stdout:216: finishing write tid 5 to smithi00140323-49 2021-01-15T19:08:49.218 INFO:tasks.rados.rados.0.smithi001.stdout:216: finishing write tid 6 to smithi00140323-49 2021-01-15T19:08:49.218 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 38 (ObjNum 107 snap 22 seq_num 107) dirty exists 2021-01-15T19:08:49.219 INFO:tasks.rados.rados.0.smithi001.stdout:216: left oid 49 (ObjNum 107 snap 22 seq_num 107) 2021-01-15T19:08:49.219 INFO:tasks.rados.rados.0.smithi001.stdout:217: expect (ObjNum 91 snap 16 seq_num 91) 2021-01-15T19:08:49.260 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:49.258+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:49.307 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:49.307 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 41/21 ECSubWrite(tid=87, reqid=client.4418.0:313, at_version=41'14, trim_to=0'0, roll_forward_to=40'10))", 2021-01-15T19:08:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.100086009999998, 2021-01-15T19:08:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998, 2021-01-15T19:08:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204611+0000", 2021-01-15T19:08:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0902000000000002e-05 2021-01-15T19:08:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204662+0000", 2021-01-15T19:08:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8193e-05 2021-01-15T19:08:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204680+0000", 2021-01-15T19:08:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7958e-05 2021-01-15T19:08:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204698+0000", 2021-01-15T19:08:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6860000000000006e-06 2021-01-15T19:08:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204706+0000", 2021-01-15T19:08:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2994000000000002e-05 2021-01-15T19:08:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204749+0000", 2021-01-15T19:08:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1491999999999999e-05 2021-01-15T19:08:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204760+0000", 2021-01-15T19:08:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056169770000000004 2021-01-15T19:08:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210377+0000", 2021-01-15T19:08:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8433999999999997e-05 2021-01-15T19:08:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210425+0000", 2021-01-15T19:08:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058146359999999998 2021-01-15T19:08:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.081097584999998, 2021-01-15T19:08:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=44, reqid=client.4418.0:334, at_version=41'25, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.079403368000001, 2021-01-15T19:08:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001, 2021-01-15T19:08:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225293+0000", 2021-01-15T19:08:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9133000000000004e-05 2021-01-15T19:08:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225363+0000", 2021-01-15T19:08:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2021-01-15T19:08:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225376+0000", 2021-01-15T19:08:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0949e-05 2021-01-15T19:08:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225387+0000", 2021-01-15T19:08:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7710000000000001e-06 2021-01-15T19:08:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225394+0000", 2021-01-15T19:08:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5732999999999997e-05 2021-01-15T19:08:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225440+0000", 2021-01-15T19:08:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5984000000000001e-05 2021-01-15T19:08:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225456+0000", 2021-01-15T19:08:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059787429999999999 2021-01-15T19:08:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231434+0000", 2021-01-15T19:08:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2653000000000001e-05 2021-01-15T19:08:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231457+0000", 2021-01-15T19:08:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061635850000000001 2021-01-15T19:08:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 47/21 ECSubWrite(tid=101, reqid=client.4418.0:384, at_version=47'23, trim_to=0'0, roll_forward_to=44'21))", 2021-01-15T19:08:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.06357627, 2021-01-15T19:08:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002, 2021-01-15T19:08:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029587999999999999 2021-01-15T19:08:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241416+0000", 2021-01-15T19:08:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6582000000000001e-05 2021-01-15T19:08:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241433+0000", 2021-01-15T19:08:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2021-01-15T19:08:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241444+0000", 2021-01-15T19:08:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.63e-06 2021-01-15T19:08:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241451+0000", 2021-01-15T19:08:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6316999999999997e-05 2021-01-15T19:08:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241497+0000", 2021-01-15T19:08:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241506+0000", 2021-01-15T19:08:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058582230000000001 2021-01-15T19:08:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247365+0000", 2021-01-15T19:08:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6705e-05 2021-01-15T19:08:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247421+0000", 2021-01-15T19:08:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002 2021-01-15T19:08:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=50'50, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.123357949000001, 2021-01-15T19:08:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996, 2021-01-15T19:08:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9509999999999999e-05 2021-01-15T19:08:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181388+0000", 2021-01-15T19:08:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.141e-05 2021-01-15T19:08:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181410+0000", 2021-01-15T19:08:49.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5201000000000001e-05 2021-01-15T19:08:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181425+0000", 2021-01-15T19:08:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5440000000000004e-06 2021-01-15T19:08:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181432+0000", 2021-01-15T19:08:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029235099999999999 2021-01-15T19:08:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181724+0000", 2021-01-15T19:08:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1324e-05 2021-01-15T19:08:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181735+0000", 2021-01-15T19:08:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005959865 2021-01-15T19:08:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187695+0000", 2021-01-15T19:08:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4237999999999999e-05 2021-01-15T19:08:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187739+0000", 2021-01-15T19:08:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996 2021-01-15T19:08:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=73, reqid=osd.5.0:73, at_version=50'52, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.122944267999999, 2021-01-15T19:08:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003, 2021-01-15T19:08:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:49.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7224000000000001e-05 2021-01-15T19:08:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181800+0000", 2021-01-15T19:08:49.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5681999999999999e-05 2021-01-15T19:08:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181825+0000", 2021-01-15T19:08:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5474000000000001e-05 2021-01-15T19:08:49.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181841+0000", 2021-01-15T19:08:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8179999999999993e-06 2021-01-15T19:08:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181850+0000", 2021-01-15T19:08:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150703 2021-01-15T19:08:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182000+0000", 2021-01-15T19:08:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2021-01-15T19:08:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182010+0000", 2021-01-15T19:08:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066268780000000001 2021-01-15T19:08:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188637+0000", 2021-01-15T19:08:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6959000000000002e-05 2021-01-15T19:08:49.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188684+0000", 2021-01-15T19:08:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003 2021-01-15T19:08:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=152, reqid=osd.4.0:152, at_version=50'30, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.119927615, 2021-01-15T19:08:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998, 2021-01-15T19:08:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1204e-05 2021-01-15T19:08:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184820+0000", 2021-01-15T19:08:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1126000000000001e-05 2021-01-15T19:08:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184842+0000", 2021-01-15T19:08:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5197999999999999e-05 2021-01-15T19:08:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184857+0000", 2021-01-15T19:08:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.336e-06 2021-01-15T19:08:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184863+0000", 2021-01-15T19:08:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2200000000000002e-05 2021-01-15T19:08:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184915+0000", 2021-01-15T19:08:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999e-05 2021-01-15T19:08:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184927+0000", 2021-01-15T19:08:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065179089999999997 2021-01-15T19:08:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191445+0000", 2021-01-15T19:08:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2212999999999997e-05 2021-01-15T19:08:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191487+0000", 2021-01-15T19:08:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998 2021-01-15T19:08:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=153, reqid=osd.4.0:153, at_version=50'32, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.119785265999999, 2021-01-15T19:08:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999, 2021-01-15T19:08:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0540000000000001e-05 2021-01-15T19:08:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184962+0000", 2021-01-15T19:08:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1452999999999999e-05 2021-01-15T19:08:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184984+0000", 2021-01-15T19:08:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4532000000000001e-05 2021-01-15T19:08:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184998+0000", 2021-01-15T19:08:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9069999999999998e-06 2021-01-15T19:08:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.185003+0000", 2021-01-15T19:08:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027679219999999999 2021-01-15T19:08:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187771+0000", 2021-01-15T19:08:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1089e-05 2021-01-15T19:08:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187782+0000", 2021-01-15T19:08:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004853792 2021-01-15T19:08:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192636+0000", 2021-01-15T19:08:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0936000000000003e-05 2021-01-15T19:08:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192667+0000", 2021-01-15T19:08:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999 2021-01-15T19:08:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:08:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.119975553, 2021-01-15T19:08:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:08:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:08:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:08:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:08:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:08:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:08:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:08:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:08:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:08:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:08:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:08:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:08:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:08:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:08:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:08:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:08:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:08:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:439 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.040686198, 2021-01-15T19:08:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007, 2021-01-15T19:08:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2021-01-15T19:08:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6384000000000001e-05 2021-01-15T19:08:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264067+0000", 2021-01-15T19:08:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7919999999999998e-06 2021-01-15T19:08:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264076+0000", 2021-01-15T19:08:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2021-01-15T19:08:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264092+0000", 2021-01-15T19:08:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7889999999999995e-06 2021-01-15T19:08:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264100+0000", 2021-01-15T19:08:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3668999999999997e-05 2021-01-15T19:08:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264154+0000", 2021-01-15T19:08:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010867800000000001 2021-01-15T19:08:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264263+0000", 2021-01-15T19:08:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008062049 2021-01-15T19:08:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272325+0000", 2021-01-15T19:08:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007 2021-01-15T19:08:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:440 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.040548545, 2021-01-15T19:08:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695, 2021-01-15T19:08:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 440 2021-01-15T19:08:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1341000000000002e-05 2021-01-15T19:08:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264200+0000", 2021-01-15T19:08:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8589999999999999e-06 2021-01-15T19:08:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264206+0000", 2021-01-15T19:08:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1953000000000001e-05 2021-01-15T19:08:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264218+0000", 2021-01-15T19:08:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.276e-06 2021-01-15T19:08:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264224+0000", 2021-01-15T19:08:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000377647 2021-01-15T19:08:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264601+0000", 2021-01-15T19:08:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8468999999999996e-05 2021-01-15T19:08:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264690+0000", 2021-01-15T19:08:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095141500000000007 2021-01-15T19:08:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274204+0000", 2021-01-15T19:08:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695 2021-01-15T19:08:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:441 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.040428086, 2021-01-15T19:08:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001, 2021-01-15T19:08:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 441 2021-01-15T19:08:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0819000000000001e-05 2021-01-15T19:08:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264340+0000", 2021-01-15T19:08:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3950000000000001e-06 2021-01-15T19:08:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264348+0000", 2021-01-15T19:08:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2213e-05 2021-01-15T19:08:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264360+0000", 2021-01-15T19:08:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6470000000000001e-06 2021-01-15T19:08:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264367+0000", 2021-01-15T19:08:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041851429999999997 2021-01-15T19:08:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268552+0000", 2021-01-15T19:08:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108403 2021-01-15T19:08:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268660+0000", 2021-01-15T19:08:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006812706 2021-01-15T19:08:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275473+0000", 2021-01-15T19:08:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001 2021-01-15T19:08:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.039018103, 2021-01-15T19:08:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:08:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:08:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:08:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:08:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:08:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:08:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:08:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:08:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:08:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:08:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:08:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:08:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:08:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:08:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:08:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:08:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:08:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:08:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.034606828999999, 2021-01-15T19:08:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:08:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:08:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:08:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:08:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:08:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:08:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:08:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:08:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:08:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:08:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:08:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:08:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:08:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:08:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:08:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:08:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:08:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:08:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=106, reqid=client.4418.0:436, at_version=51'42, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.031498685000001, 2021-01-15T19:08:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049, 2021-01-15T19:08:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184005 2021-01-15T19:08:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273382+0000", 2021-01-15T19:08:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3705999999999999e-05 2021-01-15T19:08:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273416+0000", 2021-01-15T19:08:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1336999999999999e-05 2021-01-15T19:08:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273437+0000", 2021-01-15T19:08:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4830000000000005e-06 2021-01-15T19:08:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273447+0000", 2021-01-15T19:08:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001793602 2021-01-15T19:08:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275240+0000", 2021-01-15T19:08:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3903999999999999e-05 2021-01-15T19:08:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275254+0000", 2021-01-15T19:08:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071874139999999996 2021-01-15T19:08:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282442+0000", 2021-01-15T19:08:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7597999999999999e-05 2021-01-15T19:08:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282469+0000", 2021-01-15T19:08:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049 2021-01-15T19:08:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=107, reqid=client.4418.0:437, at_version=51'43, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.030333968000001, 2021-01-15T19:08:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003, 2021-01-15T19:08:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9589999999999998e-05 2021-01-15T19:08:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274412+0000", 2021-01-15T19:08:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0508000000000001e-05 2021-01-15T19:08:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274433+0000", 2021-01-15T19:08:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7353000000000001e-05 2021-01-15T19:08:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274450+0000", 2021-01-15T19:08:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0870000000000001e-06 2021-01-15T19:08:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274457+0000", 2021-01-15T19:08:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027924999999999998 2021-01-15T19:08:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277250+0000", 2021-01-15T19:08:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3828e-05 2021-01-15T19:08:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277264+0000", 2021-01-15T19:08:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052077039999999996 2021-01-15T19:08:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282471+0000", 2021-01-15T19:08:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3986999999999999e-05 2021-01-15T19:08:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282495+0000", 2021-01-15T19:08:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003 2021-01-15T19:08:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 51/21 ECSubWrite(tid=159, reqid=client.4418.0:448, at_version=51'34, trim_to=0'0, roll_forward_to=50'32))", 2021-01-15T19:08:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.028083096, 2021-01-15T19:08:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568, 2021-01-15T19:08:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132867 2021-01-15T19:08:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276747+0000", 2021-01-15T19:08:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6607000000000003e-05 2021-01-15T19:08:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276793+0000", 2021-01-15T19:08:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9687e-05 2021-01-15T19:08:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276813+0000", 2021-01-15T19:08:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.374e-06 2021-01-15T19:08:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276820+0000", 2021-01-15T19:08:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6462999999999998e-05 2021-01-15T19:08:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276877+0000", 2021-01-15T19:08:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6032000000000002e-05 2021-01-15T19:08:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276893+0000", 2021-01-15T19:08:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006644418 2021-01-15T19:08:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283537+0000", 2021-01-15T19:08:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5120000000000003e-05 2021-01-15T19:08:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283572+0000", 2021-01-15T19:08:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568 2021-01-15T19:08:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:08:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0724873759999998, 2021-01-15T19:08:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:08:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:08:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:08:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:08:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:08:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:08:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:08:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:08:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:08:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:08:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:08:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:08:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:08:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:08:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:08:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:08:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:08:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:08:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:08:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:08:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:08:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:08:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:08:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:08:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0723393579999998, 2021-01-15T19:08:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:08:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:08:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:08:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:08:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:08:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:08:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:08:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:08:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:08:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:08:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:08:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:08:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:08:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:08:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:08:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:08:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:08:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:08:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:08:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:08:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:08:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:08:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:08:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:08:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 56/21 ECSubWrite(tid=176, reqid=client.4418.0:552, at_version=56'52, trim_to=0'0, roll_forward_to=55'47))", 2021-01-15T19:08:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:08:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0567382380000003, 2021-01-15T19:08:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010275405, 2021-01-15T19:08:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:08:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:08:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018021319999999999 2021-01-15T19:08:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249761+0000", 2021-01-15T19:08:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.951e-05 2021-01-15T19:08:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249790+0000", 2021-01-15T19:08:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3495000000000001e-05 2021-01-15T19:08:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249814+0000", 2021-01-15T19:08:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7139999999999999e-06 2021-01-15T19:08:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249822+0000", 2021-01-15T19:08:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.579e-05 2021-01-15T19:08:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249868+0000", 2021-01-15T19:08:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0170000000000002e-06 2021-01-15T19:08:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249877+0000", 2021-01-15T19:08:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083158209999999993 2021-01-15T19:08:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258193+0000", 2021-01-15T19:08:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0926000000000001e-05 2021-01-15T19:08:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258234+0000", 2021-01-15T19:08:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010275405 2021-01-15T19:08:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:49.624 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:49.626 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:49.624+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:49.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:49.726 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:49.724+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:49.774 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:08:49.776 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:49.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:49.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:49.827 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:49.825+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:49.928 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:49.927+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:49.963 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:08:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:08:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:08:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:08:49.967 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:08:49.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:49.979 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:49.998 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:50.019 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:50.029 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:50.027+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:50.041 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:50.061 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:50.082 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:50.107 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:50.112 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:50.113 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:50.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:50.124 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:50.126 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.126 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.128 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.128 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.128 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:50.131 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:50.130+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:50.188 INFO:tasks.rados.rados.0.smithi001.stdout:216: done (2 left) 2021-01-15T19:08:50.188 INFO:tasks.rados.rados.0.smithi001.stdout:217: done (1 left) 2021-01-15T19:08:50.189 INFO:tasks.rados.rados.0.smithi001.stdout:219: done (0 left) 2021-01-15T19:08:50.189 INFO:tasks.rados.rados.0.smithi001.stdout:220: copy_from oid 45 from oid 24 current snap is 23 2021-01-15T19:08:50.189 INFO:tasks.rados.rados.0.smithi001.stdout:221: read oid 35 snap -1 2021-01-15T19:08:50.190 INFO:tasks.rados.rados.0.smithi001.stdout:221: expect (ObjNum 88 snap 16 seq_num 88) 2021-01-15T19:08:50.190 INFO:tasks.rados.rados.0.smithi001.stdout:222: delete oid 14 current snap is 23 2021-01-15T19:08:50.200 INFO:tasks.rados.rados.0.smithi001.stdout:221: expect (ObjNum 88 snap 16 seq_num 88) 2021-01-15T19:08:50.211 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:50.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:50.212 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:08:50.232 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:50.231+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:50.244 INFO:tasks.rados.rados.0.smithi001.stdout:221: done (2 left) 2021-01-15T19:08:50.245 INFO:tasks.rados.rados.0.smithi001.stdout:222: done (1 left) 2021-01-15T19:08:50.245 INFO:tasks.rados.rados.0.smithi001.stdout:223: copy_from oid 38 from oid 48 current snap is 23 2021-01-15T19:08:50.245 INFO:tasks.rados.rados.0.smithi001.stdout:224: read oid 29 snap -1 2021-01-15T19:08:50.245 INFO:tasks.rados.rados.0.smithi001.stdout:224: expect deleted 2021-01-15T19:08:50.245 INFO:tasks.rados.rados.0.smithi001.stdout:225: snap_create 2021-01-15T19:08:50.274 INFO:tasks.rados.rados.0.smithi001.stdout:223: finishing copy_from to smithi00140323-38 2021-01-15T19:08:50.274 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 53 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:08:50.275 INFO:tasks.rados.rados.0.smithi001.stdout:223: finishing copy_from racing read to smithi00140323-38 2021-01-15T19:08:50.277 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:50.277 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:50.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:50.333 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:50.332+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:50.434 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:50.433+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:50.464 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:50.464 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:50.474 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:50.535 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:50.534+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:50.541 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:50.543 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:08:50.554 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2021-01-15T19:08:50.182303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90980352,"num_objects":95,"num_object_clones":44,"num_object_copies":283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":4,"num_read":272,"num_read_kb":66942,"num_write":409,"num_write_kb":95385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":21364736,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":884,"ondisk_log_size":884,"up":116,"acting":116,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230856,"kb_used_data":169992,"kb_used_omap":0,"kb_used_meta":60800,"kb_avail":754743864,"statfs":{"total":773094113280,"available":772857716736,"internally_reserved":0,"allocated":174071808,"data_stored":225912632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62259200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20860928,"num_objects":12,"num_object_clones":9,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":29,"num_read_kb":11266,"num_write":76,"num_write_kb":20196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":6184960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.005171"},"pg_stats":[{"pgid":"3.19","version":"51'26","reported_seq":"23","reported_epoch":"57","state":"peering","last_fresh":"2021-01-15T19:08:45.182513+0000","last_change":"2021-01-15T19:08:45.182895+0000","last_active":"2021-01-15T19:08:45.182513+0000","last_peered":"2021-01-15T19:08:44.260251+0000","last_clean":"2021-01-15T19:08:39.239332+0000","last_became_active":"2021-01-15T19:08:44.193634+0000","last_became_peered":"2021-01-15T19:08:44.193634+0000","last_unstale":"2021-01-15T19:08:45.182513+0000","last_undegraded":"2021-01-15T19:08:45.182513+0000","last_fullsized":"2021-01-15T19:08:45.182513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2281472,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":10,"num_read_kb":1468,"num_write":10,"num_write_kb":2228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2678784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,3,4,7],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.18","version":"51'34","reported_seq":"19","reported_epoch":"57","state":"peering","last_fresh":"2021-01-15T19:08:45.187940+0000","last_change":"2021-01-15T19:08:45.188147+0000","last_active":"2021-01-15T19:08:45.187940+0000","last_peered":"2021-01-15T19:08:44.221877+0000","last_clean":"2021-01-15T19:08:39.283886+0000","last_became_active":"2021-01-15T19:08:44.189111+0000","last_became_peered":"2021-01-15T19:08:44.189111+0000","last_unstale":"2021-01-15T19:08:45.187940+0000","last_undegraded":"2021-01-15T19:08:45.187940+0000","last_fullsized":"2021-01-15T19:08:45.187940+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3014656,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":305,"num_write":11,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,4,6,7],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.17","version":"50'30","reported_seq":"5","reported_epoch":"57","state":"peering","last_fresh":"2021-01-15T19:08:45.187411+0000","last_change":"2021-01-15T19:08:45.187411+0000","last_active":"2021-01-15T19:08:44.177445+0000","last_peered":"2021-01-15T19:08:38.188127+0000","last_clean":"2021-01-15T19:08:38.188127+0000","last_became_active":"2021-01-15T19:08:10.703898+0000","last_became_peered":"2021-01-15T19:08:10.703898+0000","last_unstale":"2021-01-15T19:08:45.187411+0000","last_undegraded":"2021-01-15T19:08:45.187411+0000","last_fullsized":"2021-01-15T19:08:45.187411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1519616,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":8,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.16","version":"43'30","reported_seq":"31","reported_epoch":"57","state":"peering","last_fresh":"2021-01-15T19:08:45.179823+0000","last_change":"2021-01-15T19:08:45.180121+0000","last_active":"2021-01-15T19:08:45.179823+0000","last_peered":"2021-01-15T19:08:44.356506+0000","last_clean":"2021-01-15T19:08:31.192624+0000","last_became_active":"2021-01-15T19:08:44.199472+0000","last_became_peered":"2021-01-15T19:08:44.199472+0000","last_unstale":"2021-01-15T19:08:45.179823+0000","last_undegraded":"2021-01-15T19:08:45.179823+0000","last_fullsized":"2021-01-15T19:08:45.179823+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1445888,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":207,"num_write":9,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":278528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,4,7],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"3.15","version":"51'20","reported_seq":"35","reported_epoch":"58","state":"active+clean","last_fresh":"2021-01-15T19:08:46.195797+0000","last_change":"2021-01-15T19:08:46.195797+0000","last_active":"2021-01-15T19:08:46.195797+0000","last_peered":"2021-01-15T19:08:46.195797+0000","last_clean":"2021-01-15T19:08:46.195797+0000","last_became_active":"2021-01-15T19:08:46.190760+0000","last_became_peered":"2021-01-15T19:08:46.190760+0000","last_unstale":"2021-01-15T19:08:46.195797+0000","last_undegraded":"2021-01-15T19:08:46.195797+0000","last_fullsized":"2021-01-15T19:08:46.195797+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":774144,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":15,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":696320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"3.14","version":"56'37","reported_seq":"45","reported_epoch":"58","state":"active+clean+wait","last_fresh":"2021-01-15T19:08:46.196019+0000","last_change":"2021-01-15T19:08:46.196019+0000","last_active":"2021-01-15T19:08:46.196019+0000","last_peered":"2021-01-15T19:08:46.196019+0000","last_clean":"2021-01-15T19:08:46.196019+0000","last_became_active":"2021-01-15T19:08:46.193553+0000","last_became_peered":"2021-01-15T19:08:46.193553+0000","last_unstale":"2021-01-15T19:08:46.196019+0000","last_undegraded":"2021-01-15T19:08:46.196019+0000","last_fullsized":"2021-01-15T19:08:46.196019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2433024,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4024,"num_write":10,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2801664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":"41","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.185775+0000","last_change":"2021-01-15T19:08:44.218900+0000","last_active":"2021-01-15T19:08:45.185775+0000","last_peered":"2021-01-15T19:08:45.185775+0000","last_clean":"2021-01-15T19:08:45.185775+0000","last_became_active":"2021-01-15T19:08:43.203849+0000","last_became_peered":"2021-01-15T19:08:43.203849+0000","last_unstale":"2021-01-15T19:08:45.185775+0000","last_undegraded":"2021-01-15T19:08:45.185775+0000","last_fullsized":"2021-01-15T19:08:45.185775+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"3.12","version":"56'30","reported_seq":"48","reported_epoch":"57","state":"peering","last_fresh":"2021-01-15T19:08:45.181353+0000","last_change":"2021-01-15T19:08:45.181624+0000","last_active":"2021-01-15T19:08:45.181353+0000","last_peered":"2021-01-15T19:08:44.318155+0000","last_clean":"2021-01-15T19:08:41.282299+0000","last_became_active":"2021-01-15T19:08:44.191819+0000","last_became_peered":"2021-01-15T19:08:44.191819+0000","last_unstale":"2021-01-15T19:08:45.181353+0000","last_undegraded":"2021-01-15T19:08:45.181353+0000","last_fullsized":"2021-01-15T19:08:45.181353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4321280,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":4829,"num_write":16,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":3809280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,3,4,7],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"1.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"55'32","reported_seq":"146","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.204207+0000","last_change":"2021-01-15T19:08:44.204207+0000","last_active":"2021-01-15T19:08:44.204207+0000","last_peered":"2021-01-15T19:08:44.204207+0000","last_clean":"2021-01-15T19:08:44.204207+0000","last_became_active":"2021-01-15T19:08:41.227163+0000","last_became_peered":"2021-01-15T19:08:41.227163+0000","last_unstale":"2021-01-15T19:08:44.204207+0000","last_undegraded":"2021-01-15T19:08:44.204207+0000","last_fullsized":"2021-01-15T19:08:44.204207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1445888,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":207,"num_write":8,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":"54","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.187203+0000","last_change":"2021-01-15T19:08:35.209452+0000","last_active":"2021-01-15T19:08:45.187203+0000","last_peered":"2021-01-15T19:08:45.187203+0000","last_clean":"2021-01-15T19:08:45.187203+0000","last_became_active":"2021-01-15T19:08:35.208618+0000","last_became_peered":"2021-01-15T19:08:35.208618+0000","last_unstale":"2021-01-15T19:08:45.187203+0000","last_undegraded":"2021-01-15T19:08:45.187203+0000","last_fullsized":"2021-01-15T19:08:45.187203+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"60","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.179576+0000","last_change":"2021-01-15T19:08:38.030592+0000","last_active":"2021-01-15T19:08:44.179576+0000","last_peered":"2021-01-15T19:08:44.179576+0000","last_clean":"2021-01-15T19:08:44.179576+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:08:44.179576+0000","last_undegraded":"2021-01-15T19:08:44.179576+0000","last_fullsized":"2021-01-15T19:08:44.179576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.030523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.030523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"55'37","reported_seq":"147","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.179462+0000","last_change":"2021-01-15T19:08:43.697876+0000","last_active":"2021-01-15T19:08:44.179462+0000","last_peered":"2021-01-15T19:08:44.179462+0000","last_clean":"2021-01-15T19:08:44.179462+0000","last_became_active":"2021-01-15T19:08:41.236424+0000","last_became_peered":"2021-01-15T19:08:41.236424+0000","last_unstale":"2021-01-15T19:08:44.179462+0000","last_undegraded":"2021-01-15T19:08:44.179462+0000","last_fullsized":"2021-01-15T19:08:44.179462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3190784,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":11,"num_write_kb":3872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"55'35","reported_seq":"152","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.176940+0000","last_change":"2021-01-15T19:08:43.691110+0000","last_active":"2021-01-15T19:08:44.176940+0000","last_peered":"2021-01-15T19:08:44.176940+0000","last_clean":"2021-01-15T19:08:44.176940+0000","last_became_active":"2021-01-15T19:08:41.239519+0000","last_became_peered":"2021-01-15T19:08:41.239519+0000","last_unstale":"2021-01-15T19:08:44.176940+0000","last_undegraded":"2021-01-15T19:08:44.176940+0000","last_fullsized":"2021-01-15T19:08:44.176940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2433024,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4023,"num_write":8,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"54","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.186912+0000","last_change":"2021-01-15T19:08:35.215835+0000","last_active":"2021-01-15T19:08:45.186912+0000","last_peered":"2021-01-15T19:08:45.186912+0000","last_clean":"2021-01-15T19:08:45.186912+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:08:45.186912+0000","last_undegraded":"2021-01-15T19:08:45.186912+0000","last_fullsized":"2021-01-15T19:08:45.186912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"55'43","reported_seq":"154","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.176277+0000","last_change":"2021-01-15T19:08:43.702776+0000","last_active":"2021-01-15T19:08:44.176277+0000","last_peered":"2021-01-15T19:08:44.176277+0000","last_clean":"2021-01-15T19:08:44.176277+0000","last_became_active":"2021-01-15T19:08:41.240170+0000","last_became_peered":"2021-01-15T19:08:41.240170+0000","last_unstale":"2021-01-15T19:08:44.176277+0000","last_undegraded":"2021-01-15T19:08:44.176277+0000","last_fullsized":"2021-01-15T19:08:44.176277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3489792,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":659,"num_write":24,"num_write_kb":3968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"58","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.190664+0000","last_change":"2021-01-15T19:08:43.003842+0000","last_active":"2021-01-15T19:08:45.190664+0000","last_peered":"2021-01-15T19:08:45.190664+0000","last_clean":"2021-01-15T19:08:45.190664+0000","last_became_active":"2021-01-15T19:08:35.210980+0000","last_became_peered":"2021-01-15T19:08:35.210980+0000","last_unstale":"2021-01-15T19:08:45.190664+0000","last_undegraded":"2021-01-15T19:08:45.190664+0000","last_fullsized":"2021-01-15T19:08:45.190664+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:43.003765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:43.003765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"132","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.179459+0000","last_change":"2021-01-15T19:08:43.189462+0000","last_active":"2021-01-15T19:08:44.179459+0000","last_peered":"2021-01-15T19:08:44.179459+0000","last_clean":"2021-01-15T19:08:44.179459+0000","last_became_active":"2021-01-15T19:08:41.234856+0000","last_became_peered":"2021-01-15T19:08:41.234856+0000","last_unstale":"2021-01-15T19:08:44.179459+0000","last_undegraded":"2021-01-15T19:08:44.179459+0000","last_fullsized":"2021-01-15T19:08:44.179459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":987136,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1610,"num_write":5,"num_write_kb":964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"1.0","version":"14'2","reported_seq":"53","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.181877+0000","last_change":"2021-01-15T19:07:51.929703+0000","last_active":"2021-01-15T19:08:44.181877+0000","last_peered":"2021-01-15T19:08:44.181877+0000","last_clean":"2021-01-15T19:08:44.181877+0000","last_became_active":"2021-01-15T19:07:51.929381+0000","last_became_peered":"2021-01-15T19:07:51.929381+0000","last_unstale":"2021-01-15T19:08:44.181877+0000","last_undegraded":"2021-01-15T19:08:44.181877+0000","last_fullsized":"2021-01-15T19:08:44.181877+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"54","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.185889+0000","last_change":"2021-01-15T19:08:35.211897+0000","last_active":"2021-01-15T19:08:45.185889+0000","last_peered":"2021-01-15T19:08:45.185889+0000","last_clean":"2021-01-15T19:08:45.185889+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:08:45.185889+0000","last_undegraded":"2021-01-15T19:08:45.185889+0000","last_fullsized":"2021-01-15T19:08:45.185889+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"55'64","reported_seq":"187","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.184940+0000","last_change":"2021-01-15T19:08:43.702798+0000","last_active":"2021-01-15T19:08:44.184940+0000","last_peered":"2021-01-15T19:08:44.184940+0000","last_clean":"2021-01-15T19:08:44.184940+0000","last_became_active":"2021-01-15T19:08:41.228824+0000","last_became_peered":"2021-01-15T19:08:41.228824+0000","last_unstale":"2021-01-15T19:08:44.184940+0000","last_undegraded":"2021-01-15T19:08:44.184940+0000","last_fullsized":"2021-01-15T19:08:44.184940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":64,"ondisk_log_size":64,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6291456,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":966,"num_write":18,"num_write_kb":7264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"69","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.178233+0000","last_change":"2021-01-15T19:08:38.923608+0000","last_active":"2021-01-15T19:08:45.178233+0000","last_peered":"2021-01-15T19:08:45.178233+0000","last_clean":"2021-01-15T19:08:45.178233+0000","last_became_active":"2021-01-15T19:08:35.204089+0000","last_became_peered":"2021-01-15T19:08:35.204089+0000","last_unstale":"2021-01-15T19:08:45.178233+0000","last_undegraded":"2021-01-15T19:08:45.178233+0000","last_fullsized":"2021-01-15T19:08:45.178233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.923498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.923498+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"107","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.185593+0000","last_change":"2021-01-15T19:08:43.181302+0000","last_active":"2021-01-15T19:08:45.185593+0000","last_peered":"2021-01-15T19:08:45.185593+0000","last_clean":"2021-01-15T19:08:45.185593+0000","last_became_active":"2021-01-15T19:08:41.228264+0000","last_became_peered":"2021-01-15T19:08:41.228264+0000","last_unstale":"2021-01-15T19:08:45.185593+0000","last_undegraded":"2021-01-15T19:08:45.185593+0000","last_fullsized":"2021-01-15T19:08:45.185593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"58","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.186175+0000","last_change":"2021-01-15T19:08:41.964641+0000","last_active":"2021-01-15T19:08:45.186175+0000","last_peered":"2021-01-15T19:08:45.186175+0000","last_clean":"2021-01-15T19:08:45.186175+0000","last_became_active":"2021-01-15T19:08:35.213894+0000","last_became_peered":"2021-01-15T19:08:35.213894+0000","last_unstale":"2021-01-15T19:08:45.186175+0000","last_undegraded":"2021-01-15T19:08:45.186175+0000","last_fullsized":"2021-01-15T19:08:45.186175+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.964569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.964569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"55'31","reported_seq":"141","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.181136+0000","last_change":"2021-01-15T19:08:43.698381+0000","last_active":"2021-01-15T19:08:44.181136+0000","last_peered":"2021-01-15T19:08:44.181136+0000","last_clean":"2021-01-15T19:08:44.181136+0000","last_became_active":"2021-01-15T19:08:41.230538+0000","last_became_peered":"2021-01-15T19:08:41.230538+0000","last_unstale":"2021-01-15T19:08:44.181136+0000","last_undegraded":"2021-01-15T19:08:44.181136+0000","last_fullsized":"2021-01-15T19:08:44.181136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2306048,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5559,"num_write":19,"num_write_kb":2252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"58","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.182023+0000","last_change":"2021-01-15T19:08:39.732649+0000","last_active":"2021-01-15T19:08:44.182023+0000","last_peered":"2021-01-15T19:08:44.182023+0000","last_clean":"2021-01-15T19:08:44.182023+0000","last_became_active":"2021-01-15T19:08:35.204979+0000","last_became_peered":"2021-01-15T19:08:35.204979+0000","last_unstale":"2021-01-15T19:08:44.182023+0000","last_undegraded":"2021-01-15T19:08:44.182023+0000","last_fullsized":"2021-01-15T19:08:44.182023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:39.732565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:39.732565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"56'52","reported_seq":"173","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.227639+0000","last_change":"2021-01-15T19:08:44.208742+0000","last_active":"2021-01-15T19:08:44.227639+0000","last_peered":"2021-01-15T19:08:44.227639+0000","last_clean":"2021-01-15T19:08:44.227639+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:08:44.227639+0000","last_undegraded":"2021-01-15T19:08:44.227639+0000","last_fullsized":"2021-01-15T19:08:44.227639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5365760,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":2538,"num_write":17,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":"17","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.180941+0000","last_change":"2021-01-15T19:08:37.215636+0000","last_active":"2021-01-15T19:08:44.180941+0000","last_peered":"2021-01-15T19:08:44.180941+0000","last_clean":"2021-01-15T19:08:44.180941+0000","last_became_active":"2021-01-15T19:08:37.215228+0000","last_became_peered":"2021-01-15T19:08:37.215228+0000","last_unstale":"2021-01-15T19:08:44.180941+0000","last_undegraded":"2021-01-15T19:08:44.180941+0000","last_fullsized":"2021-01-15T19:08:44.180941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"56'38","reported_seq":"158","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.189798+0000","last_change":"2021-01-15T19:08:44.217110+0000","last_active":"2021-01-15T19:08:45.189798+0000","last_peered":"2021-01-15T19:08:45.189798+0000","last_clean":"2021-01-15T19:08:45.189798+0000","last_became_active":"2021-01-15T19:08:41.238249+0000","last_became_peered":"2021-01-15T19:08:41.238249+0000","last_unstale":"2021-01-15T19:08:45.189798+0000","last_undegraded":"2021-01-15T19:08:45.189798+0000","last_fullsized":"2021-01-15T19:08:45.189798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5722112,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":11,"num_read_kb":1470,"num_write":17,"num_write_kb":5588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"18","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.185225+0000","last_change":"2021-01-15T19:08:37.217407+0000","last_active":"2021-01-15T19:08:45.185225+0000","last_peered":"2021-01-15T19:08:45.185225+0000","last_clean":"2021-01-15T19:08:45.185225+0000","last_became_active":"2021-01-15T19:08:37.216976+0000","last_became_peered":"2021-01-15T19:08:37.216976+0000","last_unstale":"2021-01-15T19:08:45.185225+0000","last_undegraded":"2021-01-15T19:08:45.185225+0000","last_fullsized":"2021-01-15T19:08:45.185225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"56'53","reported_seq":"158","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.186936+0000","last_change":"2021-01-15T19:08:44.206893+0000","last_active":"2021-01-15T19:08:45.186936+0000","last_peered":"2021-01-15T19:08:45.186936+0000","last_clean":"2021-01-15T19:08:45.186936+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:08:45.186936+0000","last_undegraded":"2021-01-15T19:08:45.186936+0000","last_fullsized":"2021-01-15T19:08:45.186936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10117120,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":14746,"num_write":28,"num_write_kb":9880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.b","version":"0'0","reported_seq":"53","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.185988+0000","last_change":"2021-01-15T19:08:35.210579+0000","last_active":"2021-01-15T19:08:45.185988+0000","last_peered":"2021-01-15T19:08:45.185988+0000","last_clean":"2021-01-15T19:08:45.185988+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:08:45.185988+0000","last_undegraded":"2021-01-15T19:08:45.185988+0000","last_fullsized":"2021-01-15T19:08:45.185988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2021-01-15T19:08:46.175552+0000","last_change":"2021-01-15T19:08:46.175552+0000","last_active":"2021-01-15T19:08:46.175552+0000","last_peered":"2021-01-15T19:08:46.175552+0000","last_clean":"2021-01-15T19:08:46.175552+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:08:46.175552+0000","last_undegraded":"2021-01-15T19:08:46.175552+0000","last_fullsized":"2021-01-15T19:08:46.175552+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:46.175552+0000","last_clean_scrub_stamp":"2021-01-15T19:08:46.175552+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"55'15","reported_seq":"127","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.178321+0000","last_change":"2021-01-15T19:08:43.689070+0000","last_active":"2021-01-15T19:08:45.178321+0000","last_peered":"2021-01-15T19:08:45.178321+0000","last_clean":"2021-01-15T19:08:45.178321+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:08:45.178321+0000","last_undegraded":"2021-01-15T19:08:45.178321+0000","last_fullsized":"2021-01-15T19:08:45.178321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":9822,"num_write":9,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":"57","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.176302+0000","last_change":"2021-01-15T19:08:40.178279+0000","last_active":"2021-01-15T19:08:44.176302+0000","last_peered":"2021-01-15T19:08:44.176302+0000","last_clean":"2021-01-15T19:08:44.176302+0000","last_became_active":"2021-01-15T19:08:37.218600+0000","last_became_peered":"2021-01-15T19:08:37.218600+0000","last_unstale":"2021-01-15T19:08:44.176302+0000","last_undegraded":"2021-01-15T19:08:44.176302+0000","last_fullsized":"2021-01-15T19:08:44.176302+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.178185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.178185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"56'73","reported_seq":"183","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.185674+0000","last_change":"2021-01-15T19:08:44.781278+0000","last_active":"2021-01-15T19:08:45.185674+0000","last_peered":"2021-01-15T19:08:45.185674+0000","last_clean":"2021-01-15T19:08:45.185674+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:08:45.185674+0000","last_undegraded":"2021-01-15T19:08:45.185674+0000","last_fullsized":"2021-01-15T19:08:45.185674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9535488,"num_objects":9,"num_object_clones":4,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":31,"num_read_kb":9187,"num_write":42,"num_write_kb":10832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.d","version":"0'0","reported_seq":"21","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.182219+0000","last_change":"2021-01-15T19:08:41.669593+0000","last_active":"2021-01-15T19:08:44.182219+0000","last_peered":"2021-01-15T19:08:44.182219+0000","last_clean":"2021-01-15T19:08:44.182219+0000","last_became_active":"2021-01-15T19:08:37.216171+0000","last_became_peered":"2021-01-15T19:08:37.216171+0000","last_unstale":"2021-01-15T19:08:44.182219+0000","last_undegraded":"2021-01-15T19:08:44.182219+0000","last_fullsized":"2021-01-15T19:08:44.182219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.669520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.669520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"56'27","reported_seq":"139","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.187076+0000","last_change":"2021-01-15T19:08:43.706380+0000","last_active":"2021-01-15T19:08:45.187076+0000","last_peered":"2021-01-15T19:08:45.187076+0000","last_clean":"2021-01-15T19:08:45.187076+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:08:45.187076+0000","last_undegraded":"2021-01-15T19:08:45.187076+0000","last_fullsized":"2021-01-15T19:08:45.187076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3121152,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":511,"num_write":34,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"54","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.176738+0000","last_change":"2021-01-15T19:08:37.216670+0000","last_active":"2021-01-15T19:08:44.176738+0000","last_peered":"2021-01-15T19:08:44.176738+0000","last_clean":"2021-01-15T19:08:44.176738+0000","last_became_active":"2021-01-15T19:08:37.216240+0000","last_became_peered":"2021-01-15T19:08:37.216240+0000","last_unstale":"2021-01-15T19:08:44.176738+0000","last_undegraded":"2021-01-15T19:08:44.176738+0000","last_fullsized":"2021-01-15T19:08:44.176738+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"56'13","reported_seq":"118","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.293486+0000","last_change":"2021-01-15T19:08:43.177697+0000","last_active":"2021-01-15T19:08:44.293486+0000","last_peered":"2021-01-15T19:08:44.293486+0000","last_clean":"2021-01-15T19:08:44.293486+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:08:44.293486+0000","last_undegraded":"2021-01-15T19:08:44.293486+0000","last_fullsized":"2021-01-15T19:08:44.293486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2539520,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":556,"num_write":9,"num_write_kb":2664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.f","version":"0'0","reported_seq":"58","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.186359+0000","last_change":"2021-01-15T19:08:37.964979+0000","last_active":"2021-01-15T19:08:45.186359+0000","last_peered":"2021-01-15T19:08:45.186359+0000","last_clean":"2021-01-15T19:08:45.186359+0000","last_became_active":"2021-01-15T19:08:37.224346+0000","last_became_peered":"2021-01-15T19:08:37.224346+0000","last_unstale":"2021-01-15T19:08:45.186359+0000","last_undegraded":"2021-01-15T19:08:45.186359+0000","last_fullsized":"2021-01-15T19:08:45.186359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"55'37","reported_seq":"143","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.186561+0000","last_change":"2021-01-15T19:08:43.690958+0000","last_active":"2021-01-15T19:08:45.186561+0000","last_peered":"2021-01-15T19:08:45.186561+0000","last_clean":"2021-01-15T19:08:45.186561+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:08:45.186561+0000","last_undegraded":"2021-01-15T19:08:45.186561+0000","last_fullsized":"2021-01-15T19:08:45.186561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5390336,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":2624,"num_write":37,"num_write_kb":5288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"10","length":"3"},{"start":"15","length":"2"},{"start":"1c","length":"1"}]},{"pgid":"2.e","version":"0'0","reported_seq":"67","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.179665+0000","last_change":"2021-01-15T19:08:41.084609+0000","last_active":"2021-01-15T19:08:44.179665+0000","last_peered":"2021-01-15T19:08:44.179665+0000","last_clean":"2021-01-15T19:08:44.179665+0000","last_became_active":"2021-01-15T19:08:37.216423+0000","last_became_peered":"2021-01-15T19:08:37.216423+0000","last_unstale":"2021-01-15T19:08:44.179665+0000","last_undegraded":"2021-01-15T19:08:44.179665+0000","last_fullsized":"2021-01-15T19:08:44.179665+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.084516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.084516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"58'44","reported_seq":"137","reported_epoch":"58","state":"active+clean+snaptrim+wait","last_fresh":"2021-01-15T19:08:46.213036+0000","last_change":"2021-01-15T19:08:46.201462+0000","last_active":"2021-01-15T19:08:46.213036+0000","last_peered":"2021-01-15T19:08:46.213036+0000","last_clean":"2021-01-15T19:08:46.213036+0000","last_became_active":"2021-01-15T19:08:46.196080+0000","last_became_peered":"2021-01-15T19:08:46.196080+0000","last_unstale":"2021-01-15T19:08:46.213036+0000","last_undegraded":"2021-01-15T19:08:46.213036+0000","last_fullsized":"2021-01-15T19:08:46.213036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":44,"ondisk_log_size":44,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":4341760,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":658,"num_write":23,"num_write_kb":4064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]},{"pgid":"2.10","version":"0'0","reported_seq":"21","reported_epoch":"56","state":"active+clean","last_fresh":"2021-01-15T19:08:44.179471+0000","last_change":"2021-01-15T19:08:42.076013+0000","last_active":"2021-01-15T19:08:44.179471+0000","last_peered":"2021-01-15T19:08:44.179471+0000","last_clean":"2021-01-15T19:08:44.179471+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:08:44.179471+0000","last_undegraded":"2021-01-15T19:08:44.179471+0000","last_fullsized":"2021-01-15T19:08:44.179471+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:08:42.075929+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"22","reported_epoch":"57","state":"active+clean","last_fresh":"2021-01-15T19:08:45.188385+0000","last_change":"2021-01-15T19:08:41.902763+0000","last_active":"2021-01-15T19:08:45.188385+0000","last_peered":"2021-01-15T19:08:45.188385+0000","last_clean":"2021-01-15T19:08:45.188385+0000","last_became_active":"2021-01-15T19:08:37.223920+0000","last_became_peered":"2021-01-15T19:08:37.223920+0000","last_unstale":"2021-01-15T19:08:45.188385+0000","last_undegraded":"2021-01-15T19:08:45.188385+0000","last_fullsized":"2021-01-15T19:08:45.188385+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.902685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.902685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'63","reported_seq":"59","reported_epoch":"57","state":"peering","last_fresh":"2021-01-15T19:08:45.181186+0000","last_change":"2021-01-15T19:08:45.181485+0000","last_active":"2021-01-15T19:08:45.181186+0000","last_peered":"2021-01-15T19:08:44.407005+0000","last_clean":"2021-01-15T19:08:41.262919+0000","last_became_active":"2021-01-15T19:08:44.199592+0000","last_became_peered":"2021-01-15T19:08:44.199592+0000","last_unstale":"2021-01-15T19:08:45.181186+0000","last_undegraded":"2021-01-15T19:08:45.181186+0000","last_fullsized":"2021-01-15T19:08:45.181186+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3588096,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":16,"num_read_kb":966,"num_write":19,"num_write_kb":4624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":7651328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,5,7],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"7","length":"1"},{"start":"11","length":"2"},{"start":"15","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":90980352,"num_objects":93,"num_object_clones":44,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":4,"num_read":272,"num_read_kb":66942,"num_write":407,"num_write_kb":95376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":21364736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162254848,"data_stored":215334912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":882,"ondisk_log_size":882,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34056,"kb_used_data":27200,"kb_used_omap":0,"kb_used_meta":6848,"kb_avail":94337784,"statfs":{"total":96636764160,"available":96601890816,"internally_reserved":0,"allocated":27852800,"data_stored":36244091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7012352},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":23364,"kb_used_data":15676,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94348476,"statfs":{"total":96636764160,"available":96612839424,"internally_reserved":0,"allocated":16052224,"data_stored":20528731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29164,"kb_used_data":21348,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94342676,"statfs":{"total":96636764160,"available":96606900224,"internally_reserved":0,"allocated":21860352,"data_stored":30568027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":22384,"kb_used_data":15208,"kb_used_omap":0,"kb_used_meta":7168,"kb_avail":94349456,"statfs":{"total":96636764160,"available":96613842944,"internally_reserved":0,"allocated":15572992,"data_stored":19540603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7340032},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":23448,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":24010752,"data_stored":32108123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34580,"kb_used_data":27212,"kb_used_omap":0,"kb_used_meta":7360,"kb_avail":94337260,"statfs":{"total":96636764160,"available":96601354240,"internally_reserved":0,"allocated":27865088,"data_stored":37354107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7536640},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36328,"kb_used_data":26720,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94335512,"statfs":{"total":96636764160,"available":96599564288,"internally_reserved":0,"allocated":27361280,"data_stored":35098203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":19012,"kb_used_data":13180,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":94352828,"statfs":{"total":96636764160,"available":96617295872,"internally_reserved":0,"allocated":13496320,"data_stored":14470747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]}]}} 2021-01-15T19:08:50.607 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:50.607 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:50.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:50.635 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:50.634+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:50.737 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:50.736+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:50.752 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:50.753 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:50.762 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:50.838 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:50.837+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:50.894 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:50.894 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:50.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:50.940 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:50.938+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:51.040 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:51.039+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:51.075 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:51.076 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:51.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:51.142 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:51.140+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:51.188 INFO:tasks.rados.rados.0.smithi001.stdout:223: done (3 left) 2021-01-15T19:08:51.188 INFO:tasks.rados.rados.0.smithi001.stdout:224: done (2 left) 2021-01-15T19:08:51.189 INFO:tasks.rados.rados.0.smithi001.stdout:225: done (1 left) 2021-01-15T19:08:51.189 INFO:tasks.rados.rados.0.smithi001.stdout:226: read oid 34 snap 21 2021-01-15T19:08:51.189 INFO:tasks.rados.rados.0.smithi001.stdout:226: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:08:51.190 INFO:tasks.rados.rados.0.smithi001.stdout:227: snap_remove snap 20 2021-01-15T19:08:51.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:08:51.229 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:51.229 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:51.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:51.280 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:51.279+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:51.343 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:51.342+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:51.413 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:51.414 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:51.424 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:51.468 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:51.464+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:51.544 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:51.543+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:51.601 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:51.601 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:51.611 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:51.644 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:51.644+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:51.745 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:51.744+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:51.748 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:51.749 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:51.758 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:51.846 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:51.844+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:51.924 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:51.924 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:51.933 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:51.977 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:51.976+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:52.047 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:52.045+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:52.058 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:52.059 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:52.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:52.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:52.182 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:52.180+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:52.196 INFO:tasks.rados.rados.0.smithi001.stdout:227: done (2 left) 2021-01-15T19:08:52.196 INFO:tasks.rados.rados.0.smithi001.stdout:228: read oid 23 snap -1 2021-01-15T19:08:52.196 INFO:tasks.rados.rados.0.smithi001.stdout:228: expect (ObjNum 22 snap 0 seq_num 22) 2021-01-15T19:08:52.197 INFO:tasks.rados.rados.0.smithi001.stdout:229: rollback oid 49 current snap is 24 2021-01-15T19:08:52.197 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 49 to 13 2021-01-15T19:08:52.197 INFO:tasks.rados.rados.0.smithi001.stdout:230: copy_from oid 19 from oid 4 current snap is 24 2021-01-15T19:08:52.197 INFO:tasks.rados.rados.0.smithi001.stdout:231: read oid 2 snap 21 2021-01-15T19:08:52.197 INFO:tasks.rados.rados.0.smithi001.stdout:231: expect (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:08:52.198 INFO:tasks.rados.rados.0.smithi001.stdout:232: snap_create 2021-01-15T19:08:52.199 INFO:tasks.rados.rados.0.smithi001.stdout:226: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:08:52.213 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:08:52.246 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:52.246 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:52.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:52.268 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:52.267+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:52.310 INFO:tasks.rados.rados.0.smithi001.stdout:228: expect (ObjNum 22 snap 0 seq_num 22) 2021-01-15T19:08:52.314 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:52.320 INFO:tasks.rados.rados.0.smithi001.stdout:231: expect (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:08:52.335 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:52.338 INFO:tasks.rados.rados.0.smithi001.stdout:229: finishing rollback tid 0 to smithi00140323-49 2021-01-15T19:08:52.338 INFO:tasks.rados.rados.0.smithi001.stdout:229: finishing rollback tid 1 to smithi00140323-49 2021-01-15T19:08:52.338 INFO:tasks.rados.rados.0.smithi001.stdout:229: finishing rollback tid 2 to smithi00140323-49 2021-01-15T19:08:52.338 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 44 (ObjNum 48 snap 0 seq_num 48) dirty exists 2021-01-15T19:08:52.350 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:52.349+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:52.355 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:52.376 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:52.380 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:52.380 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:52.390 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:52.396 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:52.417 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:52.438 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:52.451 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:52.450+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:52.459 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:52.519 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:52.519 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.4s1 36/21 ECSubWrite(tid=44, reqid=client.4418.0:240, at_version=36'14, trim_to=0'0, roll_forward_to=35'12))", 2021-01-15T19:08:52.520 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285963+0000", 2021-01-15T19:08:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.230657231999999, 2021-01-15T19:08:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079052319999999999, 2021-01-15T19:08:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:52.521 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285963+0000", 2021-01-15T19:08:52.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285963+0000", 2021-01-15T19:08:52.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000204279 2021-01-15T19:08:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286168+0000", 2021-01-15T19:08:52.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3572000000000001e-05 2021-01-15T19:08:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286191+0000", 2021-01-15T19:08:52.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7017e-05 2021-01-15T19:08:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286208+0000", 2021-01-15T19:08:52.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.088e-06 2021-01-15T19:08:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286214+0000", 2021-01-15T19:08:52.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044245899999999997 2021-01-15T19:08:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286657+0000", 2021-01-15T19:08:52.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2899e-05 2021-01-15T19:08:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286670+0000", 2021-01-15T19:08:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071481629999999999 2021-01-15T19:08:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.293818+0000", 2021-01-15T19:08:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0754999999999998e-05 2021-01-15T19:08:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.293869+0000", 2021-01-15T19:08:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079052319999999999 2021-01-15T19:08:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:08:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.230586073000001, 2021-01-15T19:08:52.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:08:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.534 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:08:52.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:52.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:08:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:08:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:08:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:08:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:08:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:08:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:08:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:08:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:08:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:08:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:08:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:52.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:08:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:08:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:08:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:08:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:08:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:08:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:08:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:08:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:08:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:08:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:08:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:08:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:08:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:08:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:52.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.298823306999999, 2021-01-15T19:08:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:08:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:08:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.554 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:08:52.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:08:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:08:52.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:08:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:08:52.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:08:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:08:52.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:08:52.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:08:52.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:08:52.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:08:52.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:52.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:08:52.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:08:52.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:08:52.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:08:52.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:08:52.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:08:52.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:08:52.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:08:52.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:08:52.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:08:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:08:52.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:08:52.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:08:52.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:08:52.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.570 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:52.570 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:52.570 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.296535265999999, 2021-01-15T19:08:52.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:08:52.570 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.571 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.571 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:08:52.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.572 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:52.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:52.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:08:52.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:08:52.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:08:52.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:08:52.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:08:52.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:08:52.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:08:52.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:08:52.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:08:52.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:08:52.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:08:52.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:08:52.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:08:52.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:08:52.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:08:52.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:08:52.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:08:52.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:08:52.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:08:52.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:08:52.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:08:52.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.584 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.585 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.293945641000001, 2021-01-15T19:08:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:08:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.587 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:08:52.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:52.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:52.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:08:52.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:08:52.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:08:52.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:08:52.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:08:52.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:08:52.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:08:52.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:08:52.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:08:52.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:08:52.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:08:52.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:08:52.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:08:52.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:08:52.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:08:52.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:08:52.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:08:52.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:08:52.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:08:52.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:08:52.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:08:52.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.601 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:335 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (56) in=7b] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:52.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:52.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.293138197000001, 2021-01-15T19:08:52.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086755649999999997, 2021-01-15T19:08:52.602 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.602 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.603 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2021-01-15T19:08:52.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.603 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:52.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:52.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2554e-05 2021-01-15T19:08:52.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223515+0000", 2021-01-15T19:08:52.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.776e-06 2021-01-15T19:08:52.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223522+0000", 2021-01-15T19:08:52.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2351000000000001e-05 2021-01-15T19:08:52.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223534+0000", 2021-01-15T19:08:52.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4670000000000001e-06 2021-01-15T19:08:52.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223541+0000", 2021-01-15T19:08:52.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028742450000000001 2021-01-15T19:08:52.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226415+0000", 2021-01-15T19:08:52.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2873000000000005e-05 2021-01-15T19:08:52.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226488+0000", 2021-01-15T19:08:52.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000392001 2021-01-15T19:08:52.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226880+0000", 2021-01-15T19:08:52.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061445900000000003 2021-01-15T19:08:52.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227494+0000", 2021-01-15T19:08:52.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045987420000000003 2021-01-15T19:08:52.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232093+0000", 2021-01-15T19:08:52.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5097000000000004e-05 2021-01-15T19:08:52.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232158+0000", 2021-01-15T19:08:52.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086755649999999997 2021-01-15T19:08:52.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:52.616 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:52.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.293037363, 2021-01-15T19:08:52.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:08:52.617 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.617 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:52.617 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.618 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:08:52.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.618 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:52.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:52.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:08:52.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:08:52.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:08:52.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:08:52.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:08:52.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:08:52.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:08:52.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:08:52.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:08:52.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:08:52.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:08:52.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:52.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:08:52.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:08:52.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:08:52.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:08:52.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:08:52.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:08:52.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:08:52.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:08:52.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.630 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:52.631 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:52.631 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.324353456000001, 2021-01-15T19:08:52.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:08:52.631 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.632 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.632 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:08:52.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.633 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:52.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:52.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:08:52.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:08:52.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:08:52.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:08:52.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:08:52.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:08:52.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:08:52.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:08:52.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:08:52.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:08:52.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:08:52.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:52.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:08:52.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:08:52.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:08:52.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:08:52.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:08:52.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:08:52.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:08:52.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:08:52.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:08:52.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:08:52.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:08:52.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:08:52.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:08:52.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:08:52.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.648 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:52.648 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:52.648 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.324203719, 2021-01-15T19:08:52.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:08:52.649 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.649 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:52.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.650 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:08:52.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.650 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:52.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:52.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:08:52.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:08:52.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:08:52.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:08:52.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:08:52.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:08:52.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:08:52.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:08:52.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:08:52.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:08:52.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:08:52.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:52.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:08:52.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:08:52.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:08:52.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:08:52.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:52.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:08:52.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:08:52.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:08:52.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:08:52.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:52.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:08:52.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:08:52.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:08:52.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:08:52.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:08:52.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:08:52.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:08:52.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:08:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.324051821000001, 2021-01-15T19:08:52.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:08:52.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.674 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:08:52.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:52.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:08:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:08:52.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:08:52.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:08:52.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:08:52.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:08:52.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:08:52.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:08:52.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:08:52.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:08:52.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:08:52.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:52.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:08:52.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:08:52.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:08:52.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:08:52.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:08:52.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:08:52.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:08:52.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:08:52.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:08:52.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:08:52.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:08:52.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:08:52.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:08:52.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:08:52.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.688 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:52.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:52.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.323926509, 2021-01-15T19:08:52.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:08:52.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:52.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.691 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:08:52.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:52.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:52.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:08:52.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:08:52.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:08:52.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:08:52.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:08:52.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:08:52.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:08:52.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:08:52.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:08:52.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:08:52.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:08:52.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:52.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:08:52.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:08:52.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:08:52.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:08:52.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:52.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:08:52.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:08:52.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:08:52.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:08:52.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:52.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:08:52.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:08:52.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:08:52.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:08:52.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:08:52.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:08:52.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:08:52.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:08:52.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:08:52.708 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:52.708 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.303093934, 2021-01-15T19:08:52.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:08:52.708 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.708 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.709 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:08:52.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.709 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:52.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:52.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:08:52.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:08:52.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:08:52.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:08:52.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:08:52.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:08:52.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:08:52.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:08:52.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:08:52.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:08:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:08:52.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:08:52.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:08:52.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:08:52.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:08:52.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:08:52.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:08:52.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:08:52.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:08:52.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:08:52.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:08:52.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:08:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.296800602000001, 2021-01-15T19:08:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:08:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:52.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.724 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:08:52.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.725 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:52.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:52.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:08:52.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:08:52.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:08:52.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:08:52.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:08:52.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:08:52.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:08:52.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:08:52.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:08:52.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:08:52.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:08:52.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:52.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:08:52.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:08:52.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:08:52.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:08:52.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:08:52.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:08:52.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:08:52.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:08:52.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.736 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.737 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:08:52.737 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:52.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.334741089, 2021-01-15T19:08:52.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:08:52.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.738 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:08:52.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.739 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:52.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:52.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:08:52.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:08:52.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:08:52.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:08:52.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:08:52.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:08:52.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:08:52.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:08:52.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:08:52.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:08:52.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:08:52.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:52.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:08:52.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:08:52.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:08:52.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:08:52.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:08:52.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:08:52.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:08:52.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:08:52.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:08:52.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:08:52.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:08:52.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:08:52.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:08:52.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:08:52.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.753 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.754 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:52.754 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:52.754 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.272369109, 2021-01-15T19:08:52.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:08:52.754 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.755 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.755 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.755 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:08:52.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.756 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:52.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:52.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:08:52.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:08:52.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:08:52.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:08:52.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:08:52.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:08:52.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:08:52.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:08:52.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:08:52.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:08:52.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:08:52.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:08:52.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:08:52.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:08:52.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:08:52.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:08:52.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:08:52.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:08:52.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:08:52.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:08:52.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:08:52.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.768 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.769 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:52.769 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:52.769 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.269066607999999, 2021-01-15T19:08:52.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:08:52.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.770 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.770 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:08:52.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:52.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:52.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:08:52.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:08:52.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:08:52.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:08:52.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:08:52.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:08:52.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:08:52.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:08:52.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:08:52.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:08:52.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:08:52.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:08:52.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:08:52.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:08:52.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:08:52.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:08:52.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:08:52.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:08:52.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:08:52.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:08:52.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:08:52.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.783 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.784 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:52.784 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:52.784 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.267744742, 2021-01-15T19:08:52.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:08:52.785 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.785 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.785 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.785 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.785 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.786 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:08:52.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.786 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:52.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:52.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:08:52.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:08:52.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:08:52.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:08:52.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:08:52.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:08:52.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:08:52.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:08:52.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:08:52.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:08:52.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:08:52.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:08:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:08:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:08:52.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:08:52.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:08:52.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:08:52.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:08:52.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:08:52.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:08:52.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:08:52.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:52.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:52.799 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.267374489, 2021-01-15T19:08:52.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:08:52.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:52.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.801 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:08:52.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:52.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:52.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:08:52.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:08:52.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:08:52.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:08:52.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:08:52.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:08:52.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:08:52.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:08:52.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:08:52.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:08:52.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:08:52.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:08:52.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:08:52.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:52.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:08:52.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:08:52.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:52.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:08:52.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:08:52.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:52.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:08:52.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:08:52.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:08:52.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:08:52.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.813 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:08:52.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:52.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.267040096000001, 2021-01-15T19:08:52.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:08:52.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:52.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:52.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:52.816 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:52.816 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:08:52.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.816 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:52.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:52.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:08:52.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:08:52.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:08:52.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:08:52.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:08:52.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:08:52.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:08:52.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:08:52.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:08:52.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:08:52.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:08:52.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:52.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:08:52.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:08:52.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:08:52.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:08:52.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:08:52.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:08:52.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:08:52.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:08:52.827 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.827 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.828 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.3.13:2 3.0s0 3:0c0290e4:::smithi00140323-22:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:52.828 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:52.829 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.265690532000001, 2021-01-15T19:08:52.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082549809999999998, 2021-01-15T19:08:52.829 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:52.829 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:52.829 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:52.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:52.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:52.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:52.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:52.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:52.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9704999999999997e-05 2021-01-15T19:08:52.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:52.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251010+0000", 2021-01-15T19:08:52.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5044e-05 2021-01-15T19:08:52.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:52.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251025+0000", 2021-01-15T19:08:52.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.844e-05 2021-01-15T19:08:52.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:52.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251043+0000", 2021-01-15T19:08:52.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1319999999999996e-06 2021-01-15T19:08:52.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:52.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251050+0000", 2021-01-15T19:08:52.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029025309999999999 2021-01-15T19:08:52.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:52.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253953+0000", 2021-01-15T19:08:52.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010406199999999999 2021-01-15T19:08:52.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:52.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254057+0000", 2021-01-15T19:08:52.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005128067 2021-01-15T19:08:52.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:52.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:52.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:52.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259185+0000", 2021-01-15T19:08:52.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082549809999999998 2021-01-15T19:08:52.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:52.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:52.845 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:52.846 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:52.847 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:52.845+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:52.947 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:52.946+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:52.978 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:52.979 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:52.987 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:53.048 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:53.046+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:53.112 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:53.113 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:53.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:53.166 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:53.165+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:53.180 INFO:tasks.rados.rados.0.smithi001.stdout:226: done (6 left) 2021-01-15T19:08:53.180 INFO:tasks.rados.rados.0.smithi001.stdout:228: done (5 left) 2021-01-15T19:08:53.181 INFO:tasks.rados.rados.0.smithi001.stdout:229: done (4 left) 2021-01-15T19:08:53.181 INFO:tasks.rados.rados.0.smithi001.stdout:231: done (3 left) 2021-01-15T19:08:53.181 INFO:tasks.rados.rados.0.smithi001.stdout:232: done (2 left) 2021-01-15T19:08:53.181 INFO:tasks.rados.rados.0.smithi001.stdout:233: copy_from oid 21 from oid 46 current snap is 25 2021-01-15T19:08:53.182 INFO:tasks.rados.rados.0.smithi001.stdout:234: copy_from oid 40 from oid 41 current snap is 25 2021-01-15T19:08:53.182 INFO:tasks.rados.rados.0.smithi001.stdout:235: delete oid 16 current snap is 25 2021-01-15T19:08:53.192 INFO:tasks.rados.rados.0.smithi001.stdout:235: done (4 left) 2021-01-15T19:08:53.192 INFO:tasks.rados.rados.0.smithi001.stdout:236: rollback oid 18 current snap is 25 2021-01-15T19:08:53.192 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 18 to 22 2021-01-15T19:08:53.193 INFO:tasks.rados.rados.0.smithi001.stdout:237: rmattr oid 5 current snap is 25 2021-01-15T19:08:53.193 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 25 current snap is 25 2021-01-15T19:08:53.193 INFO:tasks.rados.rados.0.smithi001.stdout:238: seq_num 111 ranges {761856=663552,1425408=663552,2088960=450560,2539520=90112} 2021-01-15T19:08:53.200 INFO:tasks.rados.rados.0.smithi001.stdout:238: writing smithi00140323-25 from 761856 to 1425408 tid 1 2021-01-15T19:08:53.204 INFO:tasks.rados.rados.0.smithi001.stdout:238: writing smithi00140323-25 from 1425408 to 2088960 tid 2 2021-01-15T19:08:53.207 INFO:tasks.rados.rados.0.smithi001.stdout:238: writing smithi00140323-25 from 2088960 to 2539520 tid 3 2021-01-15T19:08:53.207 INFO:tasks.rados.rados.0.smithi001.stdout:238: writing smithi00140323-25 from 2539520 to 2629632 tid 4 2021-01-15T19:08:53.208 INFO:tasks.rados.rados.0.smithi001.stdout:239: delete oid 27 current snap is 25 2021-01-15T19:08:53.208 INFO:tasks.rados.rados.0.smithi001.stdout:236: finishing rollback tid 1 to smithi00140323-18 2021-01-15T19:08:53.209 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 18 v 19 (ObjNum 0 snap 22026 seq_num 179397632) dirty dne 2021-01-15T19:08:53.209 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 29 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:08:53.213 INFO:tasks.rados.rados.0.smithi001.stdout:236: done (7 left) 2021-01-15T19:08:53.213 INFO:tasks.rados.rados.0.smithi001.stdout:237: done (6 left) 2021-01-15T19:08:53.213 INFO:tasks.rados.rados.0.smithi001.stdout:239: done (5 left) 2021-01-15T19:08:53.213 INFO:tasks.rados.rados.0.smithi001.stdout:240: read oid 18 snap -1 2021-01-15T19:08:53.214 INFO:tasks.rados.rados.0.smithi001.stdout:240: expect deleted 2021-01-15T19:08:53.214 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 25 2021-01-15T19:08:53.214 INFO:tasks.rados.rados.0.smithi001.stdout:241: seq_num 113 ranges {0=696320,696320=434176,1130496=466944,1597440=303104} 2021-01-15T19:08:53.216 INFO:tasks.rados.rados.0.smithi001.stdout:241: writing smithi00140323-14 from 0 to 696320 tid 1 2021-01-15T19:08:53.219 INFO:tasks.rados.rados.0.smithi001.stdout:241: writing smithi00140323-14 from 696320 to 1130496 tid 2 2021-01-15T19:08:53.222 INFO:tasks.rados.rados.0.smithi001.stdout:241: writing smithi00140323-14 from 1130496 to 1597440 tid 3 2021-01-15T19:08:53.224 INFO:tasks.rados.rados.0.smithi001.stdout:241: writing smithi00140323-14 from 1597440 to 1900544 tid 4 2021-01-15T19:08:53.224 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 12 current snap is 25 2021-01-15T19:08:53.227 INFO:tasks.rados.rados.0.smithi001.stdout:242: seq_num 114 ranges {73728=712704,786432=712704,1499136=491520} 2021-01-15T19:08:53.228 INFO:tasks.rados.rados.0.smithi001.stdout:242: writing smithi00140323-12 from 73728 to 786432 tid 1 2021-01-15T19:08:53.233 INFO:tasks.rados.rados.0.smithi001.stdout:242: writing smithi00140323-12 from 786432 to 1499136 tid 2 2021-01-15T19:08:53.238 INFO:tasks.rados.rados.0.smithi001.stdout:242: writing smithi00140323-12 from 1499136 to 1990656 tid 3 2021-01-15T19:08:53.238 INFO:tasks.rados.rados.0.smithi001.stdout:243: copy_from oid 49 from oid 36 current snap is 25 2021-01-15T19:08:53.239 INFO:tasks.rados.rados.0.smithi001.stdout:238: finishing write tid 1 to smithi00140323-25 2021-01-15T19:08:53.239 INFO:tasks.rados.rados.0.smithi001.stdout:238: finishing write tid 2 to smithi00140323-25 2021-01-15T19:08:53.239 INFO:tasks.rados.rados.0.smithi001.stdout:238: finishing write tid 3 to smithi00140323-25 2021-01-15T19:08:53.240 INFO:tasks.rados.rados.0.smithi001.stdout:238: finishing write tid 4 to smithi00140323-25 2021-01-15T19:08:53.240 INFO:tasks.rados.rados.0.smithi001.stdout:233: finishing copy_from to smithi00140323-21 2021-01-15T19:08:53.240 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 10 (ObjNum 45 snap 0 seq_num 45) dirty exists 2021-01-15T19:08:53.240 INFO:tasks.rados.rados.0.smithi001.stdout:233: finishing copy_from racing read to smithi00140323-21 2021-01-15T19:08:53.241 INFO:tasks.rados.rados.0.smithi001.stdout:238: finishing write tid 5 to smithi00140323-25 2021-01-15T19:08:53.241 INFO:tasks.rados.rados.0.smithi001.stdout:238: finishing write tid 6 to smithi00140323-25 2021-01-15T19:08:53.241 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 58 (ObjNum 111 snap 25 seq_num 111) dirty exists 2021-01-15T19:08:53.242 INFO:tasks.rados.rados.0.smithi001.stdout:238: left oid 25 (ObjNum 111 snap 25 seq_num 111) 2021-01-15T19:08:53.242 INFO:tasks.rados.rados.0.smithi001.stdout:241: finishing write tid 1 to smithi00140323-14 2021-01-15T19:08:53.242 INFO:tasks.rados.rados.0.smithi001.stdout:241: finishing write tid 2 to smithi00140323-14 2021-01-15T19:08:53.242 INFO:tasks.rados.rados.0.smithi001.stdout:241: finishing write tid 3 to smithi00140323-14 2021-01-15T19:08:53.243 INFO:tasks.rados.rados.0.smithi001.stdout:233: done (8 left) 2021-01-15T19:08:53.243 INFO:tasks.rados.rados.0.smithi001.stdout:238: done (7 left) 2021-01-15T19:08:53.243 INFO:tasks.rados.rados.0.smithi001.stdout:240: done (6 left) 2021-01-15T19:08:53.243 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 2 current snap is 25 2021-01-15T19:08:53.243 INFO:tasks.rados.rados.0.smithi001.stdout:241: finishing write tid 4 to smithi00140323-14 2021-01-15T19:08:53.244 INFO:tasks.rados.rados.0.smithi001.stdout:241: finishing write tid 5 to smithi00140323-14 2021-01-15T19:08:53.244 INFO:tasks.rados.rados.0.smithi001.stdout:244: seq_num 115 ranges {679936=729088,1409024=647168,2056192=434176,2490368=122880} 2021-01-15T19:08:53.246 INFO:tasks.rados.rados.0.smithi001.stdout:244: writing smithi00140323-2 from 679936 to 1409024 tid 1 2021-01-15T19:08:53.249 INFO:tasks.rados.rados.0.smithi001.stdout:244: writing smithi00140323-2 from 1409024 to 2056192 tid 2 2021-01-15T19:08:53.251 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:53.248+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:53.254 INFO:tasks.rados.rados.0.smithi001.stdout:244: writing smithi00140323-2 from 2056192 to 2490368 tid 3 2021-01-15T19:08:53.254 INFO:tasks.rados.rados.0.smithi001.stdout:244: writing smithi00140323-2 from 2490368 to 2613248 tid 4 2021-01-15T19:08:53.255 INFO:tasks.rados.rados.0.smithi001.stdout:245: read oid 18 snap -1 2021-01-15T19:08:53.255 INFO:tasks.rados.rados.0.smithi001.stdout:245: expect deleted 2021-01-15T19:08:53.255 INFO:tasks.rados.rados.0.smithi001.stdout:246: copy_from oid 33 from oid 34 current snap is 25 2021-01-15T19:08:53.256 INFO:tasks.rados.rados.0.smithi001.stdout:247: rollback oid 27 current snap is 25 2021-01-15T19:08:53.256 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 27 to 23 2021-01-15T19:08:53.257 INFO:tasks.rados.rados.0.smithi001.stdout:248: rollback oid 25 current snap is 25 2021-01-15T19:08:53.257 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 25 to 18 2021-01-15T19:08:53.257 INFO:tasks.rados.rados.0.smithi001.stdout:249: snap_remove snap 17 2021-01-15T19:08:53.281 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:53.281 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:53.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:53.281 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:53.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.282 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:53.282 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:53.283 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.022448320999999, 2021-01-15T19:08:53.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:53.283 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.283 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.284 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:53.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:53.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:53.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:53.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:53.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:53.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:53.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:53.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:53.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:53.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:53.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:53.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:53.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:53.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:53.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:53.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:53.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:53.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:53.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:53.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:53.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:53.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:53.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:53.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.301 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.302 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:194 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~401408] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:53.302 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:53.302 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.019721681, 2021-01-15T19:08:53.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315, 2021-01-15T19:08:53.302 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.304 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 194 2021-01-15T19:08:53.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.304 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:53.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258275+0000", 2021-01-15T19:08:53.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083910500000000004 2021-01-15T19:08:53.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259114+0000", 2021-01-15T19:08:53.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5360000000000004e-06 2021-01-15T19:08:53.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259121+0000", 2021-01-15T19:08:53.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6759e-05 2021-01-15T19:08:53.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259138+0000", 2021-01-15T19:08:53.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6370000000000005e-06 2021-01-15T19:08:53.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259147+0000", 2021-01-15T19:08:53.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017842579999999999 2021-01-15T19:08:53.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.260931+0000", 2021-01-15T19:08:53.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2162000000000004e-05 2021-01-15T19:08:53.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261013+0000", 2021-01-15T19:08:53.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049892800000000005 2021-01-15T19:08:53.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.261512+0000", 2021-01-15T19:08:53.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073416489999999996 2021-01-15T19:08:53.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268854+0000", 2021-01-15T19:08:53.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011342500000000001 2021-01-15T19:08:53.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268967+0000", 2021-01-15T19:08:53.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6855999999999997e-05 2021-01-15T19:08:53.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269044+0000", 2021-01-15T19:08:53.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010769315 2021-01-15T19:08:53.319 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.319 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.320 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.320 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:195 3.1s0 3:80f33866:::smithi00140323-14:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:53.320 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:53.321 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.018797472000003, 2021-01-15T19:08:53.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199, 2021-01-15T19:08:53.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.322 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.322 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.322 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.322 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 195 2021-01-15T19:08:53.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.323 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:53.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259199+0000", 2021-01-15T19:08:53.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1274999999999997e-05 2021-01-15T19:08:53.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259280+0000", 2021-01-15T19:08:53.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9140000000000008e-06 2021-01-15T19:08:53.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259288+0000", 2021-01-15T19:08:53.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6613999999999998e-05 2021-01-15T19:08:53.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259335+0000", 2021-01-15T19:08:53.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:08:53.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259341+0000", 2021-01-15T19:08:53.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085382489999999995 2021-01-15T19:08:53.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267879+0000", 2021-01-15T19:08:53.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5356e-05 2021-01-15T19:08:53.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.267974+0000", 2021-01-15T19:08:53.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039275900000000001 2021-01-15T19:08:53.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268367+0000", 2021-01-15T19:08:53.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010282 2021-01-15T19:08:53.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270377+0000", 2021-01-15T19:08:53.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031242999999999999 2021-01-15T19:08:53.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270690+0000", 2021-01-15T19:08:53.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8270999999999996e-05 2021-01-15T19:08:53.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.270778+0000", 2021-01-15T19:08:53.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011579199 2021-01-15T19:08:53.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.344 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.345 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:53.346 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:53.346 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.018573508999999, 2021-01-15T19:08:53.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:53.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.347 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.349 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:53.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.350 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:53.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:53.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:53.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:53.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:53.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:53.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:53.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:53.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:53.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:53.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:53.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:53.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:53.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:53.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:53.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:53.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:53.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:53.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:53.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:53.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:53.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:53.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.370 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.371 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:53.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:53.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.994214476, 2021-01-15T19:08:53.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:53.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.373 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.373 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.373 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:53.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:53.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:53.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:53.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:53.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:53.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:53.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:53.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:53.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:53.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:53.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:53.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:53.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:53.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:53.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:53.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:53.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:53.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:53.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:53.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:53.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:53.390 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:53.390 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.992035271999999, 2021-01-15T19:08:53.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:53.391 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.391 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.392 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:53.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:53.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:53.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:53.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:53.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:53.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:53.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:53.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:53.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:53.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:53.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:53.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:53.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:53.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:53.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:53.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:53.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:53.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:53.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:53.416 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.991280908, 2021-01-15T19:08:53.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:53.417 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.417 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.418 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.418 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.419 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:53.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.419 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:53.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:53.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:53.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:53.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:53.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:53.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:53.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:53.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:53.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:53.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:53.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:53.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:53.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:53.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:53.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:53.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:53.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:53.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.991160742000002, 2021-01-15T19:08:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:53.444 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:53.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:53.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:53.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:53.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:53.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:53.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:53.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:53.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.464 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.466 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:318 3.1s0 3:8b30e322:::smithi00140323-24:head [copy-from ver 8] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.081960145, 2021-01-15T19:08:53.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729, 2021-01-15T19:08:53.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.469 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.470 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 318 2021-01-15T19:08:53.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:53.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2021-01-15T19:08:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196102+0000", 2021-01-15T19:08:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6286e-05 2021-01-15T19:08:53.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196119+0000", 2021-01-15T19:08:53.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0645e-05 2021-01-15T19:08:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196139+0000", 2021-01-15T19:08:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576000000000001e-05 2021-01-15T19:08:53.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196151+0000", 2021-01-15T19:08:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017478200000000001 2021-01-15T19:08:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196326+0000", 2021-01-15T19:08:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019917200000000001 2021-01-15T19:08:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196525+0000", 2021-01-15T19:08:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099001019999999992 2021-01-15T19:08:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.206425+0000", 2021-01-15T19:08:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043897240000000002 2021-01-15T19:08:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210815+0000", 2021-01-15T19:08:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023466619999999998 2021-01-15T19:08:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213161+0000", 2021-01-15T19:08:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108811 2021-01-15T19:08:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213270+0000", 2021-01-15T19:08:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729 2021-01-15T19:08:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.500 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:319 3.1s0 3:8b30e322:::smithi00140323-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.081797226999999, 2021-01-15T19:08:53.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001, 2021-01-15T19:08:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 319 2021-01-15T19:08:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:53.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6667e-05 2021-01-15T19:08:53.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196236+0000", 2021-01-15T19:08:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196243+0000", 2021-01-15T19:08:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4992e-05 2021-01-15T19:08:53.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196258+0000", 2021-01-15T19:08:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6110000000000001e-06 2021-01-15T19:08:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196264+0000", 2021-01-15T19:08:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046740399999999999 2021-01-15T19:08:53.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196731+0000", 2021-01-15T19:08:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1105000000000004e-05 2021-01-15T19:08:53.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196812+0000", 2021-01-15T19:08:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011315451000000001 2021-01-15T19:08:53.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208128+0000", 2021-01-15T19:08:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010873 2021-01-15T19:08:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:53.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208237+0000", 2021-01-15T19:08:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058576419999999997 2021-01-15T19:08:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214094+0000", 2021-01-15T19:08:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119562 2021-01-15T19:08:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214214+0000", 2021-01-15T19:08:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012588000000000001 2021-01-15T19:08:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214340+0000", 2021-01-15T19:08:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001 2021-01-15T19:08:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:494 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:53.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.094783788000001, 2021-01-15T19:08:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1107716759999999, 2021-01-15T19:08:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 494 2021-01-15T19:08:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.092e-05 2021-01-15T19:08:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183264+0000", 2021-01-15T19:08:53.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2300000000000008e-06 2021-01-15T19:08:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183272+0000", 2021-01-15T19:08:53.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6365e-05 2021-01-15T19:08:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183288+0000", 2021-01-15T19:08:53.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3640000000000001e-06 2021-01-15T19:08:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183295+0000", 2021-01-15T19:08:53.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020715730000000002 2021-01-15T19:08:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185367+0000", 2021-01-15T19:08:53.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.627e-06 2021-01-15T19:08:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185370+0000", 2021-01-15T19:08:53.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0069268849999999 2021-01-15T19:08:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.192297+0000", 2021-01-15T19:08:53.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036647600000000002 2021-01-15T19:08:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.192663+0000", 2021-01-15T19:08:53.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.095695180000000005 2021-01-15T19:08:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.288358+0000", 2021-01-15T19:08:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112618 2021-01-15T19:08:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.288471+0000", 2021-01-15T19:08:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055134379999999998 2021-01-15T19:08:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.293984+0000", 2021-01-15T19:08:53.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1107716759999999 2021-01-15T19:08:53.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.551 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.554 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:495 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:53.554 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:53.555 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.093100079999999, 2021-01-15T19:08:53.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111341001, 2021-01-15T19:08:53.556 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.556 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.558 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 495 2021-01-15T19:08:53.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.558 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:53.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:53.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4907999999999997e-05 2021-01-15T19:08:53.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184951+0000", 2021-01-15T19:08:53.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9130000000000003e-06 2021-01-15T19:08:53.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184959+0000", 2021-01-15T19:08:53.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4218e-05 2021-01-15T19:08:53.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184973+0000", 2021-01-15T19:08:53.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.212e-06 2021-01-15T19:08:53.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184980+0000", 2021-01-15T19:08:53.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000418804 2021-01-15T19:08:53.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185398+0000", 2021-01-15T19:08:53.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2021-01-15T19:08:53.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:53.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185399+0000", 2021-01-15T19:08:53.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.007701593 2021-01-15T19:08:53.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193101+0000", 2021-01-15T19:08:53.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010404 2021-01-15T19:08:53.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193205+0000", 2021-01-15T19:08:53.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099598102999999993 2021-01-15T19:08:53.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.292803+0000", 2021-01-15T19:08:53.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122609 2021-01-15T19:08:53.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.292925+0000", 2021-01-15T19:08:53.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033118269999999998 2021-01-15T19:08:53.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.296237+0000", 2021-01-15T19:08:53.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111341001 2021-01-15T19:08:53.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.589 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:496 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:53.590 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.092975622999999, 2021-01-15T19:08:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1128853430000001, 2021-01-15T19:08:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.591 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 496 2021-01-15T19:08:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5720000000000002e-05 2021-01-15T19:08:53.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185076+0000", 2021-01-15T19:08:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9819999999999997e-06 2021-01-15T19:08:53.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185085+0000", 2021-01-15T19:08:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3526000000000001e-05 2021-01-15T19:08:53.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185099+0000", 2021-01-15T19:08:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999998e-06 2021-01-15T19:08:53.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185105+0000", 2021-01-15T19:08:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036220000000000002 2021-01-15T19:08:53.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185467+0000", 2021-01-15T19:08:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:08:53.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185469+0000", 2021-01-15T19:08:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.007797008 2021-01-15T19:08:53.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193266+0000", 2021-01-15T19:08:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102037 2021-01-15T19:08:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193368+0000", 2021-01-15T19:08:53.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10068078 2021-01-15T19:08:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.294049+0000", 2021-01-15T19:08:53.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010173 2021-01-15T19:08:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.294151+0000", 2021-01-15T19:08:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037554530000000002 2021-01-15T19:08:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.297906+0000", 2021-01-15T19:08:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1128853430000001 2021-01-15T19:08:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.617 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:507 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:08:53.618 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.045587845, 2021-01-15T19:08:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013129708, 2021-01-15T19:08:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2021-01-15T19:08:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.620 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7067000000000003e-05 2021-01-15T19:08:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232446+0000", 2021-01-15T19:08:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7929999999999992e-06 2021-01-15T19:08:53.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232455+0000", 2021-01-15T19:08:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1325000000000001e-05 2021-01-15T19:08:53.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232467+0000", 2021-01-15T19:08:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6519999999999997e-06 2021-01-15T19:08:53.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232471+0000", 2021-01-15T19:08:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006329187 2021-01-15T19:08:53.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.238801+0000", 2021-01-15T19:08:53.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010693569999999999 2021-01-15T19:08:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.239870+0000", 2021-01-15T19:08:53.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011900960000000001 2021-01-15T19:08:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.241060+0000", 2021-01-15T19:08:53.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012403300000000001 2021-01-15T19:08:53.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.241184+0000", 2021-01-15T19:08:53.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043541980000000001 2021-01-15T19:08:53.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.245538+0000", 2021-01-15T19:08:53.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013129708 2021-01-15T19:08:53.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.635 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:572 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~565248] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:53.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:53.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0270178300000001, 2021-01-15T19:08:53.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.058327720999999999, 2021-01-15T19:08:53.637 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.637 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 572 2021-01-15T19:08:53.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.639 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:53.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:53.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010807060000000001 2021-01-15T19:08:53.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252059+0000", 2021-01-15T19:08:53.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.392e-06 2021-01-15T19:08:53.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252066+0000", 2021-01-15T19:08:53.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1586999999999999e-05 2021-01-15T19:08:53.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252077+0000", 2021-01-15T19:08:53.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8189999999999999e-06 2021-01-15T19:08:53.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252082+0000", 2021-01-15T19:08:53.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4982000000000003e-05 2021-01-15T19:08:53.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252137+0000", 2021-01-15T19:08:53.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000471743 2021-01-15T19:08:53.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252609+0000", 2021-01-15T19:08:53.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045366254000000002 2021-01-15T19:08:53.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.297975+0000", 2021-01-15T19:08:53.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106475 2021-01-15T19:08:53.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.298082+0000", 2021-01-15T19:08:53.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086370999999999996 2021-01-15T19:08:53.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.298945+0000", 2021-01-15T19:08:53.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068992059999999997 2021-01-15T19:08:53.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305844+0000", 2021-01-15T19:08:53.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032253609999999999 2021-01-15T19:08:53.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309070+0000", 2021-01-15T19:08:53.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236486 2021-01-15T19:08:53.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309306+0000", 2021-01-15T19:08:53.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.058327720999999999 2021-01-15T19:08:53.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.656 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:573 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~507904] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:53.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:53.658 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0232148690000002, 2021-01-15T19:08:53.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.057854161000000001, 2021-01-15T19:08:53.658 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.658 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 573 2021-01-15T19:08:53.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.660 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:53.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:53.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050420300000000005 2021-01-15T19:08:53.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255286+0000", 2021-01-15T19:08:53.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:53.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255294+0000", 2021-01-15T19:08:53.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6209999999999999e-05 2021-01-15T19:08:53.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255310+0000", 2021-01-15T19:08:53.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2570000000000001e-06 2021-01-15T19:08:53.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255317+0000", 2021-01-15T19:08:53.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5633000000000002e-05 2021-01-15T19:08:53.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255363+0000", 2021-01-15T19:08:53.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7139999999999997e-05 2021-01-15T19:08:53.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255440+0000", 2021-01-15T19:08:53.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045807546999999997 2021-01-15T19:08:53.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.301248+0000", 2021-01-15T19:08:53.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4977999999999998e-05 2021-01-15T19:08:53.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.301343+0000", 2021-01-15T19:08:53.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066354300000000005 2021-01-15T19:08:53.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.302006+0000", 2021-01-15T19:08:53.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056809900000000003 2021-01-15T19:08:53.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307687+0000", 2021-01-15T19:08:53.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045007709999999998 2021-01-15T19:08:53.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312188+0000", 2021-01-15T19:08:53.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044770900000000002 2021-01-15T19:08:53.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312636+0000", 2021-01-15T19:08:53.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.057854161000000001 2021-01-15T19:08:53.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.676 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:574 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~507904] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:53.676 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:53.676 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.019433824, 2021-01-15T19:08:53.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055246712000000003, 2021-01-15T19:08:53.677 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.677 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 574 2021-01-15T19:08:53.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.678 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:53.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:53.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000591451 2021-01-15T19:08:53.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259154+0000", 2021-01-15T19:08:53.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9889999999999998e-06 2021-01-15T19:08:53.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259163+0000", 2021-01-15T19:08:53.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4970999999999999e-05 2021-01-15T19:08:53.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259178+0000", 2021-01-15T19:08:53.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.798e-06 2021-01-15T19:08:53.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259185+0000", 2021-01-15T19:08:53.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022518479999999999 2021-01-15T19:08:53.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261437+0000", 2021-01-15T19:08:53.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107135 2021-01-15T19:08:53.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261544+0000", 2021-01-15T19:08:53.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041508633000000003 2021-01-15T19:08:53.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.303052+0000", 2021-01-15T19:08:53.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136876 2021-01-15T19:08:53.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.303189+0000", 2021-01-15T19:08:53.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000916328 2021-01-15T19:08:53.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.304106+0000", 2021-01-15T19:08:53.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052131690000000001 2021-01-15T19:08:53.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309319+0000", 2021-01-15T19:08:53.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041306750000000003 2021-01-15T19:08:53.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313449+0000", 2021-01-15T19:08:53.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035983899999999998 2021-01-15T19:08:53.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313809+0000", 2021-01-15T19:08:53.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055246712000000003 2021-01-15T19:08:53.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:575 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~188416] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:53.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:53.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0178675669999997, 2021-01-15T19:08:53.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054232529000000002, 2021-01-15T19:08:53.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 575 2021-01-15T19:08:53.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.696 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:53.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:53.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000227791 2021-01-15T19:08:53.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260357+0000", 2021-01-15T19:08:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2290000000000003e-06 2021-01-15T19:08:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260365+0000", 2021-01-15T19:08:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6650000000000002e-05 2021-01-15T19:08:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260382+0000", 2021-01-15T19:08:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1879999999999996e-06 2021-01-15T19:08:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260389+0000", 2021-01-15T19:08:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001199133 2021-01-15T19:08:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261588+0000", 2021-01-15T19:08:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4250999999999999e-05 2021-01-15T19:08:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261642+0000", 2021-01-15T19:08:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044223854999999999 2021-01-15T19:08:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305866+0000", 2021-01-15T19:08:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8614000000000001e-05 2021-01-15T19:08:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305965+0000", 2021-01-15T19:08:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060702 2021-01-15T19:08:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.306572+0000", 2021-01-15T19:08:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006392488 2021-01-15T19:08:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312964+0000", 2021-01-15T19:08:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010513219999999999 2021-01-15T19:08:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314015+0000", 2021-01-15T19:08:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034598799999999998 2021-01-15T19:08:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314361+0000", 2021-01-15T19:08:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054232529000000002 2021-01-15T19:08:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:576 3.12s0 3:4e4266df:::smithi00140323-37:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0175631349999996, 2021-01-15T19:08:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054314145000000001, 2021-01-15T19:08:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 576 2021-01-15T19:08:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:53.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5556000000000002e-05 2021-01-15T19:08:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260479+0000", 2021-01-15T19:08:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0480000000000004e-06 2021-01-15T19:08:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260485+0000", 2021-01-15T19:08:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3729999999999993e-06 2021-01-15T19:08:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260493+0000", 2021-01-15T19:08:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9958000000000001e-05 2021-01-15T19:08:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260513+0000", 2021-01-15T19:08:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00117347 2021-01-15T19:08:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261687+0000", 2021-01-15T19:08:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5226000000000003e-05 2021-01-15T19:08:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261742+0000", 2021-01-15T19:08:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045966993999999997 2021-01-15T19:08:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307709+0000", 2021-01-15T19:08:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3752999999999995e-05 2021-01-15T19:08:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307803+0000", 2021-01-15T19:08:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057255800000000003 2021-01-15T19:08:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308375+0000", 2021-01-15T19:08:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046299920000000003 2021-01-15T19:08:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313005+0000", 2021-01-15T19:08:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001483086 2021-01-15T19:08:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314488+0000", 2021-01-15T19:08:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025913100000000001 2021-01-15T19:08:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314747+0000", 2021-01-15T19:08:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054314145000000001 2021-01-15T19:08:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:577 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:08:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0174421989999995, 2021-01-15T19:08:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056902372999999999, 2021-01-15T19:08:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 577 2021-01-15T19:08:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0988000000000001e-05 2021-01-15T19:08:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260595+0000", 2021-01-15T19:08:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260602+0000", 2021-01-15T19:08:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2503e-05 2021-01-15T19:08:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260614+0000", 2021-01-15T19:08:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7730000000000002e-06 2021-01-15T19:08:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260621+0000", 2021-01-15T19:08:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011648749999999999 2021-01-15T19:08:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261786+0000", 2021-01-15T19:08:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111148 2021-01-15T19:08:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261897+0000", 2021-01-15T19:08:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046774885000000002 2021-01-15T19:08:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308672+0000", 2021-01-15T19:08:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014140499999999999 2021-01-15T19:08:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308813+0000", 2021-01-15T19:08:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063629660000000003 2021-01-15T19:08:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.315176+0000", 2021-01-15T19:08:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014743699999999999 2021-01-15T19:08:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.315324+0000", 2021-01-15T19:08:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021328549999999999 2021-01-15T19:08:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.317457+0000", 2021-01-15T19:08:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056902372999999999 2021-01-15T19:08:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:53.766 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:53.767 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:53.766+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:53.768 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:53.868 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:53.867+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:53.934 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:53.934 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:53.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:53.970 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:53.968+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:54.071 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:54.070+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:54.108 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:54.108 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:54.109 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:54.109 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:54.109 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:54.118 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:54.172 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:54.171+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:54.187 INFO:tasks.rados.rados.0.smithi001.stdout:249: done (11 left) 2021-01-15T19:08:54.187 INFO:tasks.rados.rados.0.smithi001.stdout:250: snap_create 2021-01-15T19:08:54.187 INFO:tasks.rados.rados.0.smithi001.stdout:241: finishing write tid 6 to smithi00140323-14 2021-01-15T19:08:54.188 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 52 (ObjNum 113 snap 25 seq_num 113) dirty exists 2021-01-15T19:08:54.188 INFO:tasks.rados.rados.0.smithi001.stdout:241: left oid 14 (ObjNum 113 snap 25 seq_num 113) 2021-01-15T19:08:54.188 INFO:tasks.rados.rados.0.smithi001.stdout:242: finishing write tid 1 to smithi00140323-12 2021-01-15T19:08:54.188 INFO:tasks.rados.rados.0.smithi001.stdout:242: finishing write tid 2 to smithi00140323-12 2021-01-15T19:08:54.189 INFO:tasks.rados.rados.0.smithi001.stdout:242: finishing write tid 3 to smithi00140323-12 2021-01-15T19:08:54.189 INFO:tasks.rados.rados.0.smithi001.stdout:242: finishing write tid 4 to smithi00140323-12 2021-01-15T19:08:54.189 INFO:tasks.rados.rados.0.smithi001.stdout:242: finishing write tid 5 to smithi00140323-12 2021-01-15T19:08:54.189 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 18 (ObjNum 114 snap 25 seq_num 114) dirty exists 2021-01-15T19:08:54.189 INFO:tasks.rados.rados.0.smithi001.stdout:242: left oid 12 (ObjNum 114 snap 25 seq_num 114) 2021-01-15T19:08:54.190 INFO:tasks.rados.rados.0.smithi001.stdout:243: finishing copy_from to smithi00140323-49 2021-01-15T19:08:54.190 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2021-01-15T19:08:54.190 INFO:tasks.rados.rados.0.smithi001.stdout:243: finishing copy_from racing read to smithi00140323-49 2021-01-15T19:08:54.190 INFO:tasks.rados.rados.0.smithi001.stdout:244: finishing write tid 1 to smithi00140323-2 2021-01-15T19:08:54.190 INFO:tasks.rados.rados.0.smithi001.stdout:244: finishing write tid 2 to smithi00140323-2 2021-01-15T19:08:54.191 INFO:tasks.rados.rados.0.smithi001.stdout:247: finishing rollback tid 1 to smithi00140323-27 2021-01-15T19:08:54.191 INFO:tasks.rados.rados.0.smithi001.stdout:247: finishing rollback tid 2 to smithi00140323-27 2021-01-15T19:08:54.191 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 27 v 39 (ObjNum 37 snap 0 seq_num 37) dirty exists 2021-01-15T19:08:54.191 INFO:tasks.rados.rados.0.smithi001.stdout:244: finishing write tid 3 to smithi00140323-2 2021-01-15T19:08:54.191 INFO:tasks.rados.rados.0.smithi001.stdout:244: finishing write tid 4 to smithi00140323-2 2021-01-15T19:08:54.192 INFO:tasks.rados.rados.0.smithi001.stdout:248: finishing rollback tid 0 to smithi00140323-25 2021-01-15T19:08:54.192 INFO:tasks.rados.rados.0.smithi001.stdout:248: finishing rollback tid 1 to smithi00140323-25 2021-01-15T19:08:54.192 INFO:tasks.rados.rados.0.smithi001.stdout:244: finishing write tid 5 to smithi00140323-2 2021-01-15T19:08:54.192 INFO:tasks.rados.rados.0.smithi001.stdout:244: finishing write tid 6 to smithi00140323-2 2021-01-15T19:08:54.192 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 35 (ObjNum 115 snap 25 seq_num 115) dirty exists 2021-01-15T19:08:54.193 INFO:tasks.rados.rados.0.smithi001.stdout:244: left oid 2 (ObjNum 115 snap 25 seq_num 115) 2021-01-15T19:08:54.193 INFO:tasks.rados.rados.0.smithi001.stdout:248: finishing rollback tid 2 to smithi00140323-25 2021-01-15T19:08:54.193 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 61 (ObjNum 45 snap 0 seq_num 45) dirty exists 2021-01-15T19:08:54.193 INFO:tasks.rados.rados.0.smithi001.stdout:246: finishing copy_from to smithi00140323-33 2021-01-15T19:08:54.193 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:08:54.194 INFO:tasks.rados.rados.0.smithi001.stdout:246: finishing copy_from racing read to smithi00140323-33 2021-01-15T19:08:54.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:08:54.254 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:54.254 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:54.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:54.255 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:54.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.255 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:54.255 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:54.255 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.028400236, 2021-01-15T19:08:54.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:54.256 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.256 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.256 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:54.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:54.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:54.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:54.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:54.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:54.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:54.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:54.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:54.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:54.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:54.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:54.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:54.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:54.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:54.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:54.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:54.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:54.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:54.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.272 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.273 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 47/21 ECSubWrite(tid=101, reqid=client.4418.0:384, at_version=47'23, trim_to=0'0, roll_forward_to=44'21))", 2021-01-15T19:08:54.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:54.274 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.010878921, 2021-01-15T19:08:54.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002, 2021-01-15T19:08:54.274 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.275 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.275 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:54.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241121+0000", 2021-01-15T19:08:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029587999999999999 2021-01-15T19:08:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241416+0000", 2021-01-15T19:08:54.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6582000000000001e-05 2021-01-15T19:08:54.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241433+0000", 2021-01-15T19:08:54.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1263e-05 2021-01-15T19:08:54.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241444+0000", 2021-01-15T19:08:54.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.63e-06 2021-01-15T19:08:54.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241451+0000", 2021-01-15T19:08:54.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6316999999999997e-05 2021-01-15T19:08:54.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241497+0000", 2021-01-15T19:08:54.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:54.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.241506+0000", 2021-01-15T19:08:54.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058582230000000001 2021-01-15T19:08:54.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247365+0000", 2021-01-15T19:08:54.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6705e-05 2021-01-15T19:08:54.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.247421+0000", 2021-01-15T19:08:54.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063007890000000002 2021-01-15T19:08:54.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.294 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.295 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=50'50, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:54.295 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:54.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.0706606, 2021-01-15T19:08:54.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996, 2021-01-15T19:08:54.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:54.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:54.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9509999999999999e-05 2021-01-15T19:08:54.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181388+0000", 2021-01-15T19:08:54.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.141e-05 2021-01-15T19:08:54.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181410+0000", 2021-01-15T19:08:54.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5201000000000001e-05 2021-01-15T19:08:54.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181425+0000", 2021-01-15T19:08:54.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5440000000000004e-06 2021-01-15T19:08:54.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181432+0000", 2021-01-15T19:08:54.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029235099999999999 2021-01-15T19:08:54.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181724+0000", 2021-01-15T19:08:54.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1324e-05 2021-01-15T19:08:54.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181735+0000", 2021-01-15T19:08:54.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005959865 2021-01-15T19:08:54.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187695+0000", 2021-01-15T19:08:54.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4237999999999999e-05 2021-01-15T19:08:54.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187739+0000", 2021-01-15T19:08:54.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996 2021-01-15T19:08:54.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=73, reqid=osd.5.0:73, at_version=50'52, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:54.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:54.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.070246918999999, 2021-01-15T19:08:54.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003, 2021-01-15T19:08:54.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.311 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:54.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:54.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7224000000000001e-05 2021-01-15T19:08:54.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181800+0000", 2021-01-15T19:08:54.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5681999999999999e-05 2021-01-15T19:08:54.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181825+0000", 2021-01-15T19:08:54.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5474000000000001e-05 2021-01-15T19:08:54.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181841+0000", 2021-01-15T19:08:54.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8179999999999993e-06 2021-01-15T19:08:54.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181850+0000", 2021-01-15T19:08:54.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150703 2021-01-15T19:08:54.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182000+0000", 2021-01-15T19:08:54.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2021-01-15T19:08:54.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182010+0000", 2021-01-15T19:08:54.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066268780000000001 2021-01-15T19:08:54.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188637+0000", 2021-01-15T19:08:54.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6959000000000002e-05 2021-01-15T19:08:54.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188684+0000", 2021-01-15T19:08:54.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003 2021-01-15T19:08:54.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.328 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.329 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=152, reqid=osd.4.0:152, at_version=50'30, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:54.329 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:54.330 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.067230265999999, 2021-01-15T19:08:54.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998, 2021-01-15T19:08:54.330 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.330 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.330 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:54.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:54.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1204e-05 2021-01-15T19:08:54.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184820+0000", 2021-01-15T19:08:54.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1126000000000001e-05 2021-01-15T19:08:54.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184842+0000", 2021-01-15T19:08:54.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5197999999999999e-05 2021-01-15T19:08:54.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184857+0000", 2021-01-15T19:08:54.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.336e-06 2021-01-15T19:08:54.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184863+0000", 2021-01-15T19:08:54.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2200000000000002e-05 2021-01-15T19:08:54.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184915+0000", 2021-01-15T19:08:54.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999e-05 2021-01-15T19:08:54.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184927+0000", 2021-01-15T19:08:54.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065179089999999997 2021-01-15T19:08:54.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191445+0000", 2021-01-15T19:08:54.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2212999999999997e-05 2021-01-15T19:08:54.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191487+0000", 2021-01-15T19:08:54.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998 2021-01-15T19:08:54.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.341 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.341 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=153, reqid=osd.4.0:153, at_version=50'32, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:54.342 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:54.342 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.067087916999998, 2021-01-15T19:08:54.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999, 2021-01-15T19:08:54.342 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.342 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.343 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:54.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:54.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0540000000000001e-05 2021-01-15T19:08:54.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184962+0000", 2021-01-15T19:08:54.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1452999999999999e-05 2021-01-15T19:08:54.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184984+0000", 2021-01-15T19:08:54.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4532000000000001e-05 2021-01-15T19:08:54.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184998+0000", 2021-01-15T19:08:54.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9069999999999998e-06 2021-01-15T19:08:54.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.185003+0000", 2021-01-15T19:08:54.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027679219999999999 2021-01-15T19:08:54.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187771+0000", 2021-01-15T19:08:54.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1089e-05 2021-01-15T19:08:54.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187782+0000", 2021-01-15T19:08:54.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004853792 2021-01-15T19:08:54.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192636+0000", 2021-01-15T19:08:54.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0936000000000003e-05 2021-01-15T19:08:54.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192667+0000", 2021-01-15T19:08:54.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999 2021-01-15T19:08:54.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.353 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.354 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:08:54.354 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:54.354 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.067278204000001, 2021-01-15T19:08:54.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:08:54.355 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.355 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.355 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:54.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:54.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:08:54.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:08:54.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:08:54.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:08:54.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:54.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:08:54.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:08:54.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:08:54.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:08:54.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:08:54.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:08:54.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:08:54.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:08:54.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:08:54.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:08:54.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:08:54.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:08:54.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.366 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:439 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:54.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:54.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.987988849000001, 2021-01-15T19:08:54.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007, 2021-01-15T19:08:54.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:54.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:54.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:54.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:54.369 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2021-01-15T19:08:54.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.369 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:54.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6384000000000001e-05 2021-01-15T19:08:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264067+0000", 2021-01-15T19:08:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7919999999999998e-06 2021-01-15T19:08:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264076+0000", 2021-01-15T19:08:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2021-01-15T19:08:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264092+0000", 2021-01-15T19:08:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7889999999999995e-06 2021-01-15T19:08:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264100+0000", 2021-01-15T19:08:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3668999999999997e-05 2021-01-15T19:08:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264154+0000", 2021-01-15T19:08:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010867800000000001 2021-01-15T19:08:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264263+0000", 2021-01-15T19:08:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008062049 2021-01-15T19:08:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272325+0000", 2021-01-15T19:08:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007 2021-01-15T19:08:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.378 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:440 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:54.379 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.987851195999999, 2021-01-15T19:08:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695, 2021-01-15T19:08:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:54.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 440 2021-01-15T19:08:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.381 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1341000000000002e-05 2021-01-15T19:08:54.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264200+0000", 2021-01-15T19:08:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8589999999999999e-06 2021-01-15T19:08:54.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264206+0000", 2021-01-15T19:08:54.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1953000000000001e-05 2021-01-15T19:08:54.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264218+0000", 2021-01-15T19:08:54.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.276e-06 2021-01-15T19:08:54.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264224+0000", 2021-01-15T19:08:54.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000377647 2021-01-15T19:08:54.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264601+0000", 2021-01-15T19:08:54.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8468999999999996e-05 2021-01-15T19:08:54.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:54.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264690+0000", 2021-01-15T19:08:54.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095141500000000007 2021-01-15T19:08:54.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274204+0000", 2021-01-15T19:08:54.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695 2021-01-15T19:08:54.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.391 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:441 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:54.393 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.987730737, 2021-01-15T19:08:54.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001, 2021-01-15T19:08:54.393 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.393 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:54.393 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:54.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:54.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:54.394 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 441 2021-01-15T19:08:54.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.394 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:54.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:54.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0819000000000001e-05 2021-01-15T19:08:54.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264340+0000", 2021-01-15T19:08:54.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3950000000000001e-06 2021-01-15T19:08:54.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264348+0000", 2021-01-15T19:08:54.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2213e-05 2021-01-15T19:08:54.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264360+0000", 2021-01-15T19:08:54.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6470000000000001e-06 2021-01-15T19:08:54.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264367+0000", 2021-01-15T19:08:54.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041851429999999997 2021-01-15T19:08:54.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268552+0000", 2021-01-15T19:08:54.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108403 2021-01-15T19:08:54.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:54.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268660+0000", 2021-01-15T19:08:54.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006812706 2021-01-15T19:08:54.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275473+0000", 2021-01-15T19:08:54.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001 2021-01-15T19:08:54.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.404 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.405 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:54.405 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:54.405 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.986320753999999, 2021-01-15T19:08:54.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:08:54.406 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.406 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.406 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:54.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:54.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:08:54.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:08:54.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:08:54.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:08:54.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:08:54.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:08:54.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:08:54.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:08:54.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:08:54.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:08:54.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:08:54.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:08:54.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:08:54.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:08:54.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:08:54.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:08:54.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:08:54.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.417 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:54.417 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:54.417 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.981909480000001, 2021-01-15T19:08:54.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:08:54.418 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.418 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.418 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:54.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:54.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:08:54.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:08:54.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:08:54.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:08:54.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:08:54.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:08:54.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:08:54.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:08:54.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:08:54.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:08:54.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:08:54.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:08:54.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:08:54.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:08:54.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:08:54.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:08:54.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:08:54.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.429 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.430 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=106, reqid=client.4418.0:436, at_version=51'42, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:54.430 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:54.430 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.978801336, 2021-01-15T19:08:54.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049, 2021-01-15T19:08:54.431 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.431 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.431 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:54.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:54.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184005 2021-01-15T19:08:54.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273382+0000", 2021-01-15T19:08:54.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3705999999999999e-05 2021-01-15T19:08:54.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273416+0000", 2021-01-15T19:08:54.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1336999999999999e-05 2021-01-15T19:08:54.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273437+0000", 2021-01-15T19:08:54.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4830000000000005e-06 2021-01-15T19:08:54.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273447+0000", 2021-01-15T19:08:54.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001793602 2021-01-15T19:08:54.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275240+0000", 2021-01-15T19:08:54.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3903999999999999e-05 2021-01-15T19:08:54.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275254+0000", 2021-01-15T19:08:54.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071874139999999996 2021-01-15T19:08:54.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282442+0000", 2021-01-15T19:08:54.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7597999999999999e-05 2021-01-15T19:08:54.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282469+0000", 2021-01-15T19:08:54.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049 2021-01-15T19:08:54.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=107, reqid=client.4418.0:437, at_version=51'43, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:54.442 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:54.443 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.977636619, 2021-01-15T19:08:54.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003, 2021-01-15T19:08:54.443 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.443 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.443 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:54.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:54.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9589999999999998e-05 2021-01-15T19:08:54.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274412+0000", 2021-01-15T19:08:54.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0508000000000001e-05 2021-01-15T19:08:54.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274433+0000", 2021-01-15T19:08:54.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7353000000000001e-05 2021-01-15T19:08:54.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274450+0000", 2021-01-15T19:08:54.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0870000000000001e-06 2021-01-15T19:08:54.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274457+0000", 2021-01-15T19:08:54.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027924999999999998 2021-01-15T19:08:54.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277250+0000", 2021-01-15T19:08:54.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3828e-05 2021-01-15T19:08:54.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277264+0000", 2021-01-15T19:08:54.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052077039999999996 2021-01-15T19:08:54.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282471+0000", 2021-01-15T19:08:54.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3986999999999999e-05 2021-01-15T19:08:54.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282495+0000", 2021-01-15T19:08:54.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003 2021-01-15T19:08:54.456 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.457 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.458 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 51/21 ECSubWrite(tid=159, reqid=client.4418.0:448, at_version=51'34, trim_to=0'0, roll_forward_to=50'32))", 2021-01-15T19:08:54.459 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:54.459 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.975385747000001, 2021-01-15T19:08:54.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568, 2021-01-15T19:08:54.460 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.460 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:54.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:54.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132867 2021-01-15T19:08:54.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276747+0000", 2021-01-15T19:08:54.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6607000000000003e-05 2021-01-15T19:08:54.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276793+0000", 2021-01-15T19:08:54.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9687e-05 2021-01-15T19:08:54.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276813+0000", 2021-01-15T19:08:54.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.374e-06 2021-01-15T19:08:54.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276820+0000", 2021-01-15T19:08:54.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6462999999999998e-05 2021-01-15T19:08:54.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276877+0000", 2021-01-15T19:08:54.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6032000000000002e-05 2021-01-15T19:08:54.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276893+0000", 2021-01-15T19:08:54.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006644418 2021-01-15T19:08:54.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283537+0000", 2021-01-15T19:08:54.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5120000000000003e-05 2021-01-15T19:08:54.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283572+0000", 2021-01-15T19:08:54.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568 2021-01-15T19:08:54.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.481 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.481 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.482 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:08:54.482 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:54.482 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.019790027000001, 2021-01-15T19:08:54.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:08:54.483 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.483 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:54.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:54.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:54.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:54.484 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:08:54.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.484 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:54.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:54.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:08:54.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:08:54.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:08:54.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:08:54.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:08:54.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:08:54.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:08:54.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:08:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:08:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:08:54.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:08:54.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:54.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:08:54.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:08:54.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:08:54.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:08:54.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:08:54.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:08:54.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:54.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:08:54.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:08:54.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:08:54.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:08:54.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.501 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.503 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:08:54.503 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:54.503 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.019642009, 2021-01-15T19:08:54.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:08:54.504 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.504 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:54.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:54.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:54.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:54.506 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:08:54.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.507 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:54.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:08:54.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:08:54.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:08:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:08:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:08:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:08:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:08:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:08:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:08:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:08:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:08:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:08:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:08:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:08:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:08:54.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:08:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:54.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:08:54.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:08:54.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:08:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:08:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:08:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:08:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:08:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:08:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 56/21 ECSubWrite(tid=176, reqid=client.4418.0:552, at_version=56'52, trim_to=0'0, roll_forward_to=55'47))", 2021-01-15T19:08:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:08:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.004040889000001, 2021-01-15T19:08:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010275405, 2021-01-15T19:08:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:08:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:08:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018021319999999999 2021-01-15T19:08:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249761+0000", 2021-01-15T19:08:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.951e-05 2021-01-15T19:08:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249790+0000", 2021-01-15T19:08:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3495000000000001e-05 2021-01-15T19:08:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249814+0000", 2021-01-15T19:08:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7139999999999999e-06 2021-01-15T19:08:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249822+0000", 2021-01-15T19:08:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.579e-05 2021-01-15T19:08:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249868+0000", 2021-01-15T19:08:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0170000000000002e-06 2021-01-15T19:08:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249877+0000", 2021-01-15T19:08:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083158209999999993 2021-01-15T19:08:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258193+0000", 2021-01-15T19:08:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0926000000000001e-05 2021-01-15T19:08:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258234+0000", 2021-01-15T19:08:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010275405 2021-01-15T19:08:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 61/52 ECSubWrite(tid=210, reqid=client.4418.0:590, at_version=61'36, trim_to=0'0, roll_forward_to=55'32))", 2021-01-15T19:08:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:49.207749+0000", 2021-01-15T19:08:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0442508359999998, 2021-01-15T19:08:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00653923, 2021-01-15T19:08:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207749+0000", 2021-01-15T19:08:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207749+0000", 2021-01-15T19:08:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163506 2021-01-15T19:08:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207912+0000", 2021-01-15T19:08:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1702999999999999e-05 2021-01-15T19:08:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207934+0000", 2021-01-15T19:08:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4319e-05 2021-01-15T19:08:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207948+0000", 2021-01-15T19:08:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8259999999999993e-06 2021-01-15T19:08:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207956+0000", 2021-01-15T19:08:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9529e-05 2021-01-15T19:08:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.208006+0000", 2021-01-15T19:08:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9789999999999999e-06 2021-01-15T19:08:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.208014+0000", 2021-01-15T19:08:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062396880000000002 2021-01-15T19:08:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.214254+0000", 2021-01-15T19:08:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3680000000000003e-05 2021-01-15T19:08:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.214288+0000", 2021-01-15T19:08:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00653923 2021-01-15T19:08:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 65/21 ECSubWrite(tid=160, reqid=client.4418.0:666, at_version=65'75, trim_to=0'0, roll_forward_to=56'73))", 2021-01-15T19:08:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.266015+0000", 2021-01-15T19:08:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.985984737, 2021-01-15T19:08:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095205849999999998, 2021-01-15T19:08:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:54.567 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:54.567 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:54.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:54.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.266015+0000", 2021-01-15T19:08:54.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:54.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.266015+0000", 2021-01-15T19:08:54.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015126079999999999 2021-01-15T19:08:54.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267527+0000", 2021-01-15T19:08:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0474e-05 2021-01-15T19:08:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267558+0000", 2021-01-15T19:08:54.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4299e-05 2021-01-15T19:08:54.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:54.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267572+0000", 2021-01-15T19:08:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2490000000000004e-06 2021-01-15T19:08:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267579+0000", 2021-01-15T19:08:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4147999999999998e-05 2021-01-15T19:08:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:54.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267624+0000", 2021-01-15T19:08:54.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8019999999999992e-06 2021-01-15T19:08:54.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:54.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267631+0000", 2021-01-15T19:08:54.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078664819999999993 2021-01-15T19:08:54.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:54.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.275498+0000", 2021-01-15T19:08:54.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7523e-05 2021-01-15T19:08:54.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:54.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:54.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:54.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.275535+0000", 2021-01-15T19:08:54.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095205849999999998 2021-01-15T19:08:54.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.589 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:54.589 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:54.589 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:54.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:54.592 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:54.590+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:54.593 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:08:54.692 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:54.691+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:54.743 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:54.764 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:54.776 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:54.776 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:54.785 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:54.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:08:54.806 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:54.807 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:54.804+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:54.827 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:54.848 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:54.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:54.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:54.891 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.891 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.891 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.892 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.893 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:08:54.895 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:54.894+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:54.922 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:54.923 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:54.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:08:55.019 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:55.018+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:55.097 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:55.096+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:55.106 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:55.107 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:55.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:08:55.188 INFO:tasks.rados.rados.0.smithi001.stdout:241: done (11 left) 2021-01-15T19:08:55.188 INFO:tasks.rados.rados.0.smithi001.stdout:242: done (10 left) 2021-01-15T19:08:55.189 INFO:tasks.rados.rados.0.smithi001.stdout:243: done (9 left) 2021-01-15T19:08:55.189 INFO:tasks.rados.rados.0.smithi001.stdout:244: done (8 left) 2021-01-15T19:08:55.190 INFO:tasks.rados.rados.0.smithi001.stdout:245: done (7 left) 2021-01-15T19:08:55.190 INFO:tasks.rados.rados.0.smithi001.stdout:246: done (6 left) 2021-01-15T19:08:55.190 INFO:tasks.rados.rados.0.smithi001.stdout:247: done (5 left) 2021-01-15T19:08:55.191 INFO:tasks.rados.rados.0.smithi001.stdout:248: done (4 left) 2021-01-15T19:08:55.191 INFO:tasks.rados.rados.0.smithi001.stdout:250: done (3 left) 2021-01-15T19:08:55.192 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 26 2021-01-15T19:08:55.192 INFO:tasks.rados.rados.0.smithi001.stdout:251: seq_num 116 ranges {2351104=745472,3096576=532480,3629056=638976,4268032=32768} 2021-01-15T19:08:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:08:55.197 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:55.196+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:55.206 INFO:tasks.rados.rados.0.smithi001.stdout:251: writing smithi00140323-13 from 2351104 to 3096576 tid 1 2021-01-15T19:08:55.208 INFO:tasks.rados.rados.0.smithi001.stdout:251: writing smithi00140323-13 from 3096576 to 3629056 tid 2 2021-01-15T19:08:55.212 INFO:tasks.rados.rados.0.smithi001.stdout:251: writing smithi00140323-13 from 3629056 to 4268032 tid 3 2021-01-15T19:08:55.213 INFO:tasks.rados.rados.0.smithi001.stdout:251: writing smithi00140323-13 from 4268032 to 4300800 tid 4 2021-01-15T19:08:55.213 INFO:tasks.rados.rados.0.smithi001.stdout:252: rmattr oid 49 current snap is 26 2021-01-15T19:08:55.214 INFO:tasks.rados.rados.0.smithi001.stdout:252: done (4 left) 2021-01-15T19:08:55.214 INFO:tasks.rados.rados.0.smithi001.stdout:253: rmattr oid 2 current snap is 26 2021-01-15T19:08:55.214 INFO:tasks.rados.rados.0.smithi001.stdout:253: done (4 left) 2021-01-15T19:08:55.215 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 42 current snap is 26 2021-01-15T19:08:55.215 INFO:tasks.rados.rados.0.smithi001.stdout:254: seq_num 117 ranges {696320=770048,1466368=417792,1884160=442368,2326528=344064} 2021-01-15T19:08:55.220 INFO:tasks.rados.rados.0.smithi001.stdout:254: writing smithi00140323-42 from 696320 to 1466368 tid 1 2021-01-15T19:08:55.223 INFO:tasks.rados.rados.0.smithi001.stdout:254: writing smithi00140323-42 from 1466368 to 1884160 tid 2 2021-01-15T19:08:55.225 INFO:tasks.rados.rados.0.smithi001.stdout:254: writing smithi00140323-42 from 1884160 to 2326528 tid 3 2021-01-15T19:08:55.229 INFO:tasks.rados.rados.0.smithi001.stdout:254: writing smithi00140323-42 from 2326528 to 2670592 tid 4 2021-01-15T19:08:55.229 INFO:tasks.rados.rados.0.smithi001.stdout:255: read oid 22 snap -1 2021-01-15T19:08:55.229 INFO:tasks.rados.rados.0.smithi001.stdout:255: expect (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:08:55.230 INFO:tasks.rados.rados.0.smithi001.stdout:256: snap_remove snap 18 2021-01-15T19:08:55.298 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:55.297+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:55.305 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:55.305 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:55.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:08:55.399 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:08:55.398+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:08:55.487 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:55.487 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:55.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:08:55.539 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:55.537+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:55.598 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:08:55.599 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:08:55.599 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2021-01-15T19:08:55.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2021-01-15T19:08:55.602 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:55.601+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:55.670 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:55.670 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:55.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:08:55.703 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:55.702+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:55.808 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:55.807+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:55.830 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:55.831 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:55.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:08:55.909 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:55.908+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:55.920 INFO:teuthology.orchestra.run.smithi001.stdout:moved 0 / 136 (0%) 2021-01-15T19:08:55.920 INFO:teuthology.orchestra.run.smithi001.stdout:avg 17 2021-01-15T19:08:55.920 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.60555 -> 3.60555 (expected baseline 3.85681) 2021-01-15T19:08:55.921 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.7 with 11 -> 11 pgs (0.647059 -> 0.647059 * mean) 2021-01-15T19:08:55.921 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.3 with 22 -> 22 pgs (1.29412 -> 1.29412 * mean) 2021-01-15T19:08:55.921 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:55.921 INFO:teuthology.orchestra.run.smithi001.stdout:oload 1000 2021-01-15T19:08:55.921 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 1 2021-01-15T19:08:55.922 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2021-01-15T19:08:55.922 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0003 2021-01-15T19:08:55.922 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0028 2021-01-15T19:08:55.923 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2021-01-15T19:08:55.933 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2021-01-15T19:08:55.934 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2021-01-15T19:08:55.967 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:55.967 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:55.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:08:56.020 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:56.018+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:56.111 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:56.110+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:56.157 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:56.157 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:56.168 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:08:56.188 INFO:tasks.rados.rados.0.smithi001.stdout:256: done (6 left) 2021-01-15T19:08:56.189 INFO:tasks.rados.rados.0.smithi001.stdout:257: snap_remove snap 10 2021-01-15T19:08:56.211 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:56.210+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:56.294 INFO:tasks.ceph.mgr.y.smithi001.stderr:2021-01-15T19:08:56.292+0000 7f516c355700 -1 mgr.server reply reply (4) Interrupted system call no change 2021-01-15T19:08:56.294 INFO:teuthology.orchestra.run.smithi001.stdout:moved 16 / 136 (11.7647%) 2021-01-15T19:08:56.294 INFO:teuthology.orchestra.run.smithi001.stdout:avg 17 2021-01-15T19:08:56.295 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.60555 -> 4.24264 (expected baseline 3.85681) 2021-01-15T19:08:56.295 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.7 with 11 -> 8 pgs (0.647059 -> 0.470588 * mean) 2021-01-15T19:08:56.295 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.3 with 22 -> 19 pgs (1.29412 -> 1.11765 * mean) 2021-01-15T19:08:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:56.296 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2021-01-15T19:08:56.296 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 3 2021-01-15T19:08:56.297 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2021-01-15T19:08:56.297 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0003 2021-01-15T19:08:56.297 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0003 2021-01-15T19:08:56.297 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 weight 1.0000 -> 0.7793 2021-01-15T19:08:56.297 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 weight 1.0000 -> 0.8249 2021-01-15T19:08:56.298 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 weight 1.0000 -> 0.8325 2021-01-15T19:08:56.298 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 1.0000 -> 0.8420 2021-01-15T19:08:56.298 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2021-01-15T19:08:56.307 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2021-01-15T19:08:56.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2021-01-15T19:08:56.329 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:56.330 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:56.330 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:56.328+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:56.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:08:56.424 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:56.423+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:56.513 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:56.513 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:56.516 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:08:56.514+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:08:56.523 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:08:56.617 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:56.616+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:56.649 INFO:teuthology.orchestra.run.smithi001.stdout:moved 0 / 136 (0%) 2021-01-15T19:08:56.650 INFO:teuthology.orchestra.run.smithi001.stdout:avg 17 2021-01-15T19:08:56.650 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.60555 -> 3.60555 (expected baseline 3.85681) 2021-01-15T19:08:56.650 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.7 with 11 -> 11 pgs (0.647059 -> 0.647059 * mean) 2021-01-15T19:08:56.651 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.3 with 22 -> 22 pgs (1.29412 -> 1.29412 * mean) 2021-01-15T19:08:56.651 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:56.651 INFO:teuthology.orchestra.run.smithi001.stdout:oload 1000 2021-01-15T19:08:56.651 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 0.05 2021-01-15T19:08:56.652 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2021-01-15T19:08:56.652 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0003 2021-01-15T19:08:56.652 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0030 2021-01-15T19:08:56.652 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2021-01-15T19:08:56.662 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2021-01-15T19:08:56.663 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2021-01-15T19:08:56.699 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:56.699 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:56.713 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:08:56.758 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:56.757+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:56.819 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:56.818+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:56.848 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:56.849 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:56.859 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:08:56.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:08:56.943 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:56.942+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:56.980 INFO:tasks.ceph.mgr.y.smithi001.stderr:2021-01-15T19:08:56.978+0000 7f516c355700 -1 mgr.server reply reply (4) Interrupted system call no change 2021-01-15T19:08:56.980 INFO:teuthology.orchestra.run.smithi001.stdout:moved 16 / 136 (11.7647%) 2021-01-15T19:08:56.980 INFO:teuthology.orchestra.run.smithi001.stdout:avg 17 2021-01-15T19:08:56.981 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.60555 -> 4.24264 (expected baseline 3.85681) 2021-01-15T19:08:56.981 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.7 with 11 -> 8 pgs (0.647059 -> 0.470588 * mean) 2021-01-15T19:08:56.981 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.3 with 22 -> 19 pgs (1.29412 -> 1.11765 * mean) 2021-01-15T19:08:56.981 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:56.982 INFO:teuthology.orchestra.run.smithi001.stdout:oload 110 2021-01-15T19:08:56.982 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 1 2021-01-15T19:08:56.982 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2021-01-15T19:08:56.982 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0003 2021-01-15T19:08:56.983 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0003 2021-01-15T19:08:56.983 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 weight 1.0000 -> 0.7793 2021-01-15T19:08:56.983 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 weight 1.0000 -> 0.8249 2021-01-15T19:08:56.983 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 weight 1.0000 -> 0.8325 2021-01-15T19:08:56.983 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 weight 1.0000 -> 0.8420 2021-01-15T19:08:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2021-01-15T19:08:56.993 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2021-01-15T19:08:56.994 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2021-01-15T19:08:57.024 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:08:57.025 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:08:57.034 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:08:57.037 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:57.036+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:57.056 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:57.076 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:57.096 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:57.117 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:08:57.122 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:57.121+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:57.142 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:08:57.163 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:08:57.184 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:08:57.188 INFO:tasks.rados.rados.0.smithi001.stdout:257: done (6 left) 2021-01-15T19:08:57.189 INFO:tasks.rados.rados.0.smithi001.stdout:258: snap_create 2021-01-15T19:08:57.189 INFO:tasks.rados.rados.0.smithi001.stdout:251: finishing write tid 1 to smithi00140323-13 2021-01-15T19:08:57.189 INFO:tasks.rados.rados.0.smithi001.stdout:251: finishing write tid 2 to smithi00140323-13 2021-01-15T19:08:57.189 INFO:tasks.rados.rados.0.smithi001.stdout:251: finishing write tid 3 to smithi00140323-13 2021-01-15T19:08:57.190 INFO:tasks.rados.rados.0.smithi001.stdout:251: finishing write tid 4 to smithi00140323-13 2021-01-15T19:08:57.190 INFO:tasks.rados.rados.0.smithi001.stdout:251: finishing write tid 5 to smithi00140323-13 2021-01-15T19:08:57.190 INFO:tasks.rados.rados.0.smithi001.stdout:251: finishing write tid 6 to smithi00140323-13 2021-01-15T19:08:57.191 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 69 (ObjNum 116 snap 26 seq_num 116) dirty exists 2021-01-15T19:08:57.191 INFO:tasks.rados.rados.0.smithi001.stdout:251: left oid 13 (ObjNum 116 snap 26 seq_num 116) 2021-01-15T19:08:57.191 INFO:tasks.rados.rados.0.smithi001.stdout:254: finishing write tid 1 to smithi00140323-42 2021-01-15T19:08:57.191 INFO:tasks.rados.rados.0.smithi001.stdout:254: finishing write tid 2 to smithi00140323-42 2021-01-15T19:08:57.192 INFO:tasks.rados.rados.0.smithi001.stdout:254: finishing write tid 3 to smithi00140323-42 2021-01-15T19:08:57.192 INFO:tasks.rados.rados.0.smithi001.stdout:254: finishing write tid 4 to smithi00140323-42 2021-01-15T19:08:57.193 INFO:tasks.rados.rados.0.smithi001.stdout:255: expect (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:08:57.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:08:57.213 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:57.213 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:57.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:57.214 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:57.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.215 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:185 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:57.215 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.250240+0000", 2021-01-15T19:08:57.215 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.960061355999997, 2021-01-15T19:08:57.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012592686000000001, 2021-01-15T19:08:57.215 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.216 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.216 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.217 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 185 2021-01-15T19:08:57.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.217 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250240+0000", 2021-01-15T19:08:57.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250240+0000", 2021-01-15T19:08:57.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1121000000000001e-05 2021-01-15T19:08:57.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250291+0000", 2021-01-15T19:08:57.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7689999999999999e-06 2021-01-15T19:08:57.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250298+0000", 2021-01-15T19:08:57.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1284000000000001e-05 2021-01-15T19:08:57.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250309+0000", 2021-01-15T19:08:57.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8729999999999998e-06 2021-01-15T19:08:57.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250314+0000", 2021-01-15T19:08:57.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055389940000000002 2021-01-15T19:08:57.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255853+0000", 2021-01-15T19:08:57.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110605 2021-01-15T19:08:57.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255964+0000", 2021-01-15T19:08:57.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044564399999999998 2021-01-15T19:08:57.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256409+0000", 2021-01-15T19:08:57.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033779320000000002 2021-01-15T19:08:57.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259787+0000", 2021-01-15T19:08:57.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029721909999999999 2021-01-15T19:08:57.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262759+0000", 2021-01-15T19:08:57.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3273000000000001e-05 2021-01-15T19:08:57.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262833+0000", 2021-01-15T19:08:57.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012592686000000001 2021-01-15T19:08:57.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.234 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.235 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:186 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:57.235 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.250354+0000", 2021-01-15T19:08:57.235 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.95994692, 2021-01-15T19:08:57.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013953569000000001, 2021-01-15T19:08:57.236 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.236 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:57.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.236 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.237 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 186 2021-01-15T19:08:57.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.237 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250354+0000", 2021-01-15T19:08:57.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250354+0000", 2021-01-15T19:08:57.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8504000000000003e-05 2021-01-15T19:08:57.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250393+0000", 2021-01-15T19:08:57.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3960000000000003e-06 2021-01-15T19:08:57.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250399+0000", 2021-01-15T19:08:57.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1952e-05 2021-01-15T19:08:57.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250411+0000", 2021-01-15T19:08:57.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3059999999999999e-06 2021-01-15T19:08:57.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250416+0000", 2021-01-15T19:08:57.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062491630000000003 2021-01-15T19:08:57.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256665+0000", 2021-01-15T19:08:57.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101201 2021-01-15T19:08:57.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:57.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256766+0000", 2021-01-15T19:08:57.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006536717 2021-01-15T19:08:57.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263303+0000", 2021-01-15T19:08:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010614400000000001 2021-01-15T19:08:57.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263409+0000", 2021-01-15T19:08:57.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089918599999999995 2021-01-15T19:08:57.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264308+0000", 2021-01-15T19:08:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013953569000000001 2021-01-15T19:08:57.248 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:322 3.6s0 3:6feca5e1:::smithi00140323-9:head [append 0~0] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196304+0000", 2021-01-15T19:08:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.013997794000002, 2021-01-15T19:08:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011982255000000001, 2021-01-15T19:08:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.250 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 322 2021-01-15T19:08:57.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.252 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196304+0000", 2021-01-15T19:08:57.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196304+0000", 2021-01-15T19:08:57.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9557000000000001e-05 2021-01-15T19:08:57.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196343+0000", 2021-01-15T19:08:57.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0759999999999999e-06 2021-01-15T19:08:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196349+0000", 2021-01-15T19:08:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2045000000000001e-05 2021-01-15T19:08:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196361+0000", 2021-01-15T19:08:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0529999999999997e-06 2021-01-15T19:08:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196366+0000", 2021-01-15T19:08:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2932000000000002e-05 2021-01-15T19:08:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196409+0000", 2021-01-15T19:08:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5000000000000005e-06 2021-01-15T19:08:57.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:57.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196419+0000", 2021-01-15T19:08:57.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071189759999999999 2021-01-15T19:08:57.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.203538+0000", 2021-01-15T19:08:57.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117361 2021-01-15T19:08:57.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.203655+0000", 2021-01-15T19:08:57.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048490099999999998 2021-01-15T19:08:57.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.204140+0000", 2021-01-15T19:08:57.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003496201 2021-01-15T19:08:57.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.207636+0000", 2021-01-15T19:08:57.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8499999999999995e-05 2021-01-15T19:08:57.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.207735+0000", 2021-01-15T19:08:57.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055115299999999997 2021-01-15T19:08:57.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208286+0000", 2021-01-15T19:08:57.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011982255000000001 2021-01-15T19:08:57.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:364 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [append 0~344064] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:57.268 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191348+0000", 2021-01-15T19:08:57.268 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.018953628999999, 2021-01-15T19:08:57.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013923016999999999, 2021-01-15T19:08:57.268 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.268 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.269 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 364 2021-01-15T19:08:57.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.270 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191348+0000", 2021-01-15T19:08:57.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191348+0000", 2021-01-15T19:08:57.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044911899999999999 2021-01-15T19:08:57.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191797+0000", 2021-01-15T19:08:57.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.989e-06 2021-01-15T19:08:57.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191802+0000", 2021-01-15T19:08:57.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2268e-05 2021-01-15T19:08:57.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191814+0000", 2021-01-15T19:08:57.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9809999999999997e-06 2021-01-15T19:08:57.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191818+0000", 2021-01-15T19:08:57.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024894549999999998 2021-01-15T19:08:57.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.194308+0000", 2021-01-15T19:08:57.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116953 2021-01-15T19:08:57.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.194425+0000", 2021-01-15T19:08:57.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065544400000000004 2021-01-15T19:08:57.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195080+0000", 2021-01-15T19:08:57.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005784015 2021-01-15T19:08:57.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.200864+0000", 2021-01-15T19:08:57.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043187659999999999 2021-01-15T19:08:57.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205183+0000", 2021-01-15T19:08:57.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2021-01-15T19:08:57.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205271+0000", 2021-01-15T19:08:57.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013923016999999999 2021-01-15T19:08:57.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.285 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:365 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [setxattr _header (57) in=7b] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:57.285 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:08:57.286 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.018460266000002, 2021-01-15T19:08:57.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014071523000000001, 2021-01-15T19:08:57.286 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.286 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.287 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.287 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 365 2021-01-15T19:08:57.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.288 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:08:57.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:08:57.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5833000000000002e-05 2021-01-15T19:08:57.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191867+0000", 2021-01-15T19:08:57.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5580000000000001e-06 2021-01-15T19:08:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191871+0000", 2021-01-15T19:08:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0240000000000003e-06 2021-01-15T19:08:57.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191877+0000", 2021-01-15T19:08:57.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4040000000000002e-06 2021-01-15T19:08:57.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191879+0000", 2021-01-15T19:08:57.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046935880000000003 2021-01-15T19:08:57.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196573+0000", 2021-01-15T19:08:57.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107329 2021-01-15T19:08:57.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196680+0000", 2021-01-15T19:08:57.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047579799999999999 2021-01-15T19:08:57.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197156+0000", 2021-01-15T19:08:57.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003761206 2021-01-15T19:08:57.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.200917+0000", 2021-01-15T19:08:57.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044910649999999998 2021-01-15T19:08:57.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205408+0000", 2021-01-15T19:08:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050471800000000003 2021-01-15T19:08:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205913+0000", 2021-01-15T19:08:57.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014071523000000001 2021-01-15T19:08:57.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.303 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:366 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:57.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:08:57.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.018402383000002, 2021-01-15T19:08:57.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999, 2021-01-15T19:08:57.304 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:57.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.305 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 366 2021-01-15T19:08:57.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:08:57.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:08:57.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0789999999999999e-05 2021-01-15T19:08:57.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191920+0000", 2021-01-15T19:08:57.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2760000000000001e-06 2021-01-15T19:08:57.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191923+0000", 2021-01-15T19:08:57.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4840000000000001e-06 2021-01-15T19:08:57.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191930+0000", 2021-01-15T19:08:57.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2479999999999999e-06 2021-01-15T19:08:57.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191932+0000", 2021-01-15T19:08:57.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055448800000000003 2021-01-15T19:08:57.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197477+0000", 2021-01-15T19:08:57.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101512 2021-01-15T19:08:57.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:57.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197578+0000", 2021-01-15T19:08:57.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009253308 2021-01-15T19:08:57.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206832+0000", 2021-01-15T19:08:57.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011591699999999999 2021-01-15T19:08:57.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206947+0000", 2021-01-15T19:08:57.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012706729999999999 2021-01-15T19:08:57.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.208218+0000", 2021-01-15T19:08:57.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999 2021-01-15T19:08:57.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.319 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 50/21 ECSubWriteReply(tid=157, last_complete=50'18, committed=1, applied=1))", 2021-01-15T19:08:57.319 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:08:57.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.517373207999999, 2021-01-15T19:08:57.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001, 2021-01-15T19:08:57.320 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.320 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:57.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:08:57.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:08:57.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4277000000000002e-05 2021-01-15T19:08:57.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692952+0000", 2021-01-15T19:08:57.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015131444000000001 2021-01-15T19:08:57.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708084+0000", 2021-01-15T19:08:57.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9973999999999999e-05 2021-01-15T19:08:57.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708104+0000", 2021-01-15T19:08:57.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4741e-05 2021-01-15T19:08:57.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708119+0000", 2021-01-15T19:08:57.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2155000000000004e-05 2021-01-15T19:08:57.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708181+0000", 2021-01-15T19:08:57.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7632999999999999e-05 2021-01-15T19:08:57.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708198+0000", 2021-01-15T19:08:57.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001 2021-01-15T19:08:57.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.331 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:57.331 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:08:57.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.946171093, 2021-01-15T19:08:57.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:08:57.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.332 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.332 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:08:57.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.333 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:08:57.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:08:57.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:08:57.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:08:57.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:08:57.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:08:57.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:08:57.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:08:57.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:08:57.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:08:57.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:08:57.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:08:57.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:08:57.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:08:57.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:08:57.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:08:57.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:08:57.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:08:57.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:08:57.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:08:57.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:08:57.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:08:57.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:08:57.345 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.346 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.346 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:08:57.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:08:57.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.945759465999998, 2021-01-15T19:08:57.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:08:57.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.347 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:57.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.348 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:08:57.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.349 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:08:57.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:08:57.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:08:57.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:08:57.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:08:57.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:08:57.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:08:57.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:08:57.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:08:57.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:08:57.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:08:57.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:08:57.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:08:57.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:57.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:08:57.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:08:57.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:08:57.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:08:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:08:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:08:57.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:08:57.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:08:57.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:08:57.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:08:57.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:08:57.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:08:57.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.362 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.363 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:559 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:57.363 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.214034+0000", 2021-01-15T19:08:57.363 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.996267769999999, 2021-01-15T19:08:57.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011465448, 2021-01-15T19:08:57.363 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.364 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.364 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 559 2021-01-15T19:08:57.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.365 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214034+0000", 2021-01-15T19:08:57.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214034+0000", 2021-01-15T19:08:57.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0046000000000002e-05 2021-01-15T19:08:57.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214084+0000", 2021-01-15T19:08:57.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.602e-06 2021-01-15T19:08:57.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214092+0000", 2021-01-15T19:08:57.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4688999999999999e-05 2021-01-15T19:08:57.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214107+0000", 2021-01-15T19:08:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214114+0000", 2021-01-15T19:08:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057394900000000003 2021-01-15T19:08:57.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214688+0000", 2021-01-15T19:08:57.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013961600000000001 2021-01-15T19:08:57.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214828+0000", 2021-01-15T19:08:57.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076523900000000002 2021-01-15T19:08:57.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.215593+0000", 2021-01-15T19:08:57.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056695829999999997 2021-01-15T19:08:57.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.221263+0000", 2021-01-15T19:08:57.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041835090000000002 2021-01-15T19:08:57.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.225446+0000", 2021-01-15T19:08:57.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2785999999999999e-05 2021-01-15T19:08:57.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.225499+0000", 2021-01-15T19:08:57.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011465448 2021-01-15T19:08:57.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.378 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:560 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 1f] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:57.380 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.214167+0000", 2021-01-15T19:08:57.381 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.996134425999999, 2021-01-15T19:08:57.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012833993, 2021-01-15T19:08:57.381 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.381 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.382 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 560 2021-01-15T19:08:57.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.383 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214167+0000", 2021-01-15T19:08:57.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214167+0000", 2021-01-15T19:08:57.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5794000000000002e-05 2021-01-15T19:08:57.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214213+0000", 2021-01-15T19:08:57.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5909999999999998e-06 2021-01-15T19:08:57.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214220+0000", 2021-01-15T19:08:57.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3059e-05 2021-01-15T19:08:57.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214233+0000", 2021-01-15T19:08:57.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.536e-06 2021-01-15T19:08:57.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214239+0000", 2021-01-15T19:08:57.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047897110000000003 2021-01-15T19:08:57.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.219029+0000", 2021-01-15T19:08:57.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013429499999999999 2021-01-15T19:08:57.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.219163+0000", 2021-01-15T19:08:57.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046991099999999999 2021-01-15T19:08:57.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.219633+0000", 2021-01-15T19:08:57.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003649074 2021-01-15T19:08:57.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.223282+0000", 2021-01-15T19:08:57.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036660640000000001 2021-01-15T19:08:57.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.226948+0000", 2021-01-15T19:08:57.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2957999999999997e-05 2021-01-15T19:08:57.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.227001+0000", 2021-01-15T19:08:57.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012833993 2021-01-15T19:08:57.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:561 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:57.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.214286+0000", 2021-01-15T19:08:57.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.996015913999999, 2021-01-15T19:08:57.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013434122999999999, 2021-01-15T19:08:57.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 561 2021-01-15T19:08:57.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214286+0000", 2021-01-15T19:08:57.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214286+0000", 2021-01-15T19:08:57.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3198999999999998e-05 2021-01-15T19:08:57.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214329+0000", 2021-01-15T19:08:57.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:08:57.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214336+0000", 2021-01-15T19:08:57.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2753e-05 2021-01-15T19:08:57.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214348+0000", 2021-01-15T19:08:57.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5070000000000002e-06 2021-01-15T19:08:57.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214353+0000", 2021-01-15T19:08:57.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076004059999999997 2021-01-15T19:08:57.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.221953+0000", 2021-01-15T19:08:57.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103965 2021-01-15T19:08:57.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.222057+0000", 2021-01-15T19:08:57.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053538399999999997 2021-01-15T19:08:57.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.222593+0000", 2021-01-15T19:08:57.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077479700000000003 2021-01-15T19:08:57.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.223367+0000", 2021-01-15T19:08:57.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042704739999999998 2021-01-15T19:08:57.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.227638+0000", 2021-01-15T19:08:57.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1773999999999993e-05 2021-01-15T19:08:57.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.227720+0000", 2021-01-15T19:08:57.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013434122999999999 2021-01-15T19:08:57.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.418 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.419 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:588 3.6s0 3:660240bf:::smithi00140323-49:head [append 0~598016] snapc 24=[24,23,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e61)", 2021-01-15T19:08:57.419 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:49.199104+0000", 2021-01-15T19:08:57.419 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.0111973699999997, 2021-01-15T19:08:57.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011299208, 2021-01-15T19:08:57.420 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.420 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.421 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 588 2021-01-15T19:08:57.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199104+0000", 2021-01-15T19:08:57.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199104+0000", 2021-01-15T19:08:57.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065357700000000002 2021-01-15T19:08:57.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199758+0000", 2021-01-15T19:08:57.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2539999999999992e-06 2021-01-15T19:08:57.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199766+0000", 2021-01-15T19:08:57.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4070000000000001e-05 2021-01-15T19:08:57.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199780+0000", 2021-01-15T19:08:57.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7689999999999996e-06 2021-01-15T19:08:57.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199788+0000", 2021-01-15T19:08:57.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9562000000000003e-05 2021-01-15T19:08:57.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199847+0000", 2021-01-15T19:08:57.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142423 2021-01-15T19:08:57.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.199990+0000", 2021-01-15T19:08:57.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000767379 2021-01-15T19:08:57.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.200757+0000", 2021-01-15T19:08:57.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002563863 2021-01-15T19:08:57.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203321+0000", 2021-01-15T19:08:57.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007018725 2021-01-15T19:08:57.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.210340+0000", 2021-01-15T19:08:57.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3585999999999997e-05 2021-01-15T19:08:57.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.210403+0000", 2021-01-15T19:08:57.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011299208 2021-01-15T19:08:57.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.434 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:589 3.6s0 3:660240bf:::smithi00140323-49:head [append 0~729088] snapc 24=[24,23,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e61)", 2021-01-15T19:08:57.434 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:49.203151+0000", 2021-01-15T19:08:57.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.0071505009999999, 2021-01-15T19:08:57.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012388110000000001, 2021-01-15T19:08:57.435 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.436 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 589 2021-01-15T19:08:57.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203151+0000", 2021-01-15T19:08:57.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203151+0000", 2021-01-15T19:08:57.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000747052 2021-01-15T19:08:57.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203898+0000", 2021-01-15T19:08:57.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5559999999999996e-06 2021-01-15T19:08:57.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203908+0000", 2021-01-15T19:08:57.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6201999999999998e-05 2021-01-15T19:08:57.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203924+0000", 2021-01-15T19:08:57.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5369999999999997e-06 2021-01-15T19:08:57.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203932+0000", 2021-01-15T19:08:57.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9701999999999999e-05 2021-01-15T19:08:57.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203982+0000", 2021-01-15T19:08:57.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0798999999999996e-05 2021-01-15T19:08:57.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.204073+0000", 2021-01-15T19:08:57.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078019299999999999 2021-01-15T19:08:57.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.204853+0000", 2021-01-15T19:08:57.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064241489999999997 2021-01-15T19:08:57.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.211277+0000", 2021-01-15T19:08:57.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041737470000000002 2021-01-15T19:08:57.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.215451+0000", 2021-01-15T19:08:57.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8172999999999998e-05 2021-01-15T19:08:57.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.215539+0000", 2021-01-15T19:08:57.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012388110000000001 2021-01-15T19:08:57.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.449 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:634 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~663552] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:08:57.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:08:57.450 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0102311100000003, 2021-01-15T19:08:57.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952, 2021-01-15T19:08:57.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.451 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 634 2021-01-15T19:08:57.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.451 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:08:57.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:08:57.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081858500000000004 2021-01-15T19:08:57.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200889+0000", 2021-01-15T19:08:57.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2700000000000004e-06 2021-01-15T19:08:57.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200897+0000", 2021-01-15T19:08:57.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7638e-05 2021-01-15T19:08:57.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200915+0000", 2021-01-15T19:08:57.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0730000000000005e-06 2021-01-15T19:08:57.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200923+0000", 2021-01-15T19:08:57.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6312999999999999e-05 2021-01-15T19:08:57.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200999+0000", 2021-01-15T19:08:57.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018274900000000001 2021-01-15T19:08:57.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.201182+0000", 2021-01-15T19:08:57.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001367224 2021-01-15T19:08:57.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.202549+0000", 2021-01-15T19:08:57.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012523378 2021-01-15T19:08:57.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215073+0000", 2021-01-15T19:08:57.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6226999999999999e-05 2021-01-15T19:08:57.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215169+0000", 2021-01-15T19:08:57.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131495 2021-01-15T19:08:57.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215300+0000", 2021-01-15T19:08:57.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952 2021-01-15T19:08:57.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.463 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.464 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.464 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s2 65/52 ECSubWrite(tid=97, reqid=client.4418.0:627, at_version=65'33, trim_to=0'0, roll_forward_to=55'31))", 2021-01-15T19:08:57.464 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.203494+0000", 2021-01-15T19:08:57.465 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0068074820000001, 2021-01-15T19:08:57.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013481198999999999, 2021-01-15T19:08:57.465 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.465 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:57.465 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.203494+0000", 2021-01-15T19:08:57.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.203494+0000", 2021-01-15T19:08:57.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000652997 2021-01-15T19:08:57.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204147+0000", 2021-01-15T19:08:57.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1455999999999997e-05 2021-01-15T19:08:57.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204188+0000", 2021-01-15T19:08:57.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5899000000000001e-05 2021-01-15T19:08:57.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204214+0000", 2021-01-15T19:08:57.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8681999999999998e-05 2021-01-15T19:08:57.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204233+0000", 2021-01-15T19:08:57.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0841000000000003e-05 2021-01-15T19:08:57.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204284+0000", 2021-01-15T19:08:57.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1681e-05 2021-01-15T19:08:57.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204295+0000", 2021-01-15T19:08:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012651252 2021-01-15T19:08:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216947+0000", 2021-01-15T19:08:57.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8391e-05 2021-01-15T19:08:57.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216975+0000", 2021-01-15T19:08:57.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013481198999999999 2021-01-15T19:08:57.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.476 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.476 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:635 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~663552] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:08:57.477 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.204419+0000", 2021-01-15T19:08:57.477 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0058825389999999, 2021-01-15T19:08:57.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011569596, 2021-01-15T19:08:57.477 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.477 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.478 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 635 2021-01-15T19:08:57.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204419+0000", 2021-01-15T19:08:57.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204419+0000", 2021-01-15T19:08:57.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065310300000000004 2021-01-15T19:08:57.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.205072+0000", 2021-01-15T19:08:57.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8869999999999992e-06 2021-01-15T19:08:57.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.205080+0000", 2021-01-15T19:08:57.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2082999999999999e-05 2021-01-15T19:08:57.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.205092+0000", 2021-01-15T19:08:57.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1949999999999996e-06 2021-01-15T19:08:57.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.205096+0000", 2021-01-15T19:08:57.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011600110000000001 2021-01-15T19:08:57.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.206256+0000", 2021-01-15T19:08:57.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5558999999999999e-05 2021-01-15T19:08:57.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.206342+0000", 2021-01-15T19:08:57.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062451599999999998 2021-01-15T19:08:57.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.206966+0000", 2021-01-15T19:08:57.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008425419 2021-01-15T19:08:57.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215392+0000", 2021-01-15T19:08:57.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053377700000000004 2021-01-15T19:08:57.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215925+0000", 2021-01-15T19:08:57.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3046000000000006e-05 2021-01-15T19:08:57.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215988+0000", 2021-01-15T19:08:57.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011569596 2021-01-15T19:08:57.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.491 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.492 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:638 3.8s0 3:1009af39:::smithi00140323-25:head [setxattr _header (57) in=7b] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:08:57.492 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.207660+0000", 2021-01-15T19:08:57.492 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0026413740000004, 2021-01-15T19:08:57.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013508197, 2021-01-15T19:08:57.493 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.493 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:57.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.494 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2021-01-15T19:08:57.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.494 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207660+0000", 2021-01-15T19:08:57.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207660+0000", 2021-01-15T19:08:57.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9713000000000003e-05 2021-01-15T19:08:57.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207710+0000", 2021-01-15T19:08:57.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5010000000000001e-06 2021-01-15T19:08:57.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207716+0000", 2021-01-15T19:08:57.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2828e-05 2021-01-15T19:08:57.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207729+0000", 2021-01-15T19:08:57.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1819999999999997e-06 2021-01-15T19:08:57.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207734+0000", 2021-01-15T19:08:57.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0090069680000000006 2021-01-15T19:08:57.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216741+0000", 2021-01-15T19:08:57.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013609700000000001 2021-01-15T19:08:57.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216877+0000", 2021-01-15T19:08:57.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052799300000000004 2021-01-15T19:08:57.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:57.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217405+0000", 2021-01-15T19:08:57.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026540370000000001 2021-01-15T19:08:57.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:57.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.220059+0000", 2021-01-15T19:08:57.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010326790000000001 2021-01-15T19:08:57.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:57.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221092+0000", 2021-01-15T19:08:57.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6198999999999993e-05 2021-01-15T19:08:57.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221168+0000", 2021-01-15T19:08:57.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013508197 2021-01-15T19:08:57.506 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.507 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.507 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:639 3.8s0 3:1009af39:::smithi00140323-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:08:57.508 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:08:57.508 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.0025299810000003, 2021-01-15T19:08:57.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074, 2021-01-15T19:08:57.508 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.508 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:57.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.509 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 639 2021-01-15T19:08:57.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.510 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:08:57.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:08:57.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1959000000000003e-05 2021-01-15T19:08:57.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207813+0000", 2021-01-15T19:08:57.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7560000000000002e-06 2021-01-15T19:08:57.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207820+0000", 2021-01-15T19:08:57.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:08:57.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207832+0000", 2021-01-15T19:08:57.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.583e-06 2021-01-15T19:08:57.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207837+0000", 2021-01-15T19:08:57.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010014853000000001 2021-01-15T19:08:57.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217852+0000", 2021-01-15T19:08:57.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0618000000000006e-05 2021-01-15T19:08:57.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:57.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217942+0000", 2021-01-15T19:08:57.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035136210000000002 2021-01-15T19:08:57.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221456+0000", 2021-01-15T19:08:57.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116369 2021-01-15T19:08:57.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221572+0000", 2021-01-15T19:08:57.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001478222 2021-01-15T19:08:57.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.223051+0000", 2021-01-15T19:08:57.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074 2021-01-15T19:08:57.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.532 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:656 3.6s0 3:660240bf:::smithi00140323-49:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:08:57.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.236940+0000", 2021-01-15T19:08:57.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9733610700000002, 2021-01-15T19:08:57.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011083061999999999, 2021-01-15T19:08:57.536 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:57.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:57.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:57.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:57.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:57.537 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 656 2021-01-15T19:08:57.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.537 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:57.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:57.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.236940+0000", 2021-01-15T19:08:57.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:57.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:57.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.236940+0000", 2021-01-15T19:08:57.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8330000000000001e-05 2021-01-15T19:08:57.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:57.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.236979+0000", 2021-01-15T19:08:57.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6889999999999998e-06 2021-01-15T19:08:57.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:57.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.236985+0000", 2021-01-15T19:08:57.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2887e-05 2021-01-15T19:08:57.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:57.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.236998+0000", 2021-01-15T19:08:57.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.916e-06 2021-01-15T19:08:57.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:57.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.237003+0000", 2021-01-15T19:08:57.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019312299999999999 2021-01-15T19:08:57.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.237196+0000", 2021-01-15T19:08:57.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6072000000000001e-05 2021-01-15T19:08:57.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:57.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.237252+0000", 2021-01-15T19:08:57.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067839440000000001 2021-01-15T19:08:57.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.244036+0000", 2021-01-15T19:08:57.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115402 2021-01-15T19:08:57.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:57.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.244152+0000", 2021-01-15T19:08:57.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035992870000000001 2021-01-15T19:08:57.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:57.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.247751+0000", 2021-01-15T19:08:57.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128682 2021-01-15T19:08:57.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:57.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.247880+0000", 2021-01-15T19:08:57.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014373000000000001 2021-01-15T19:08:57.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:57.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:57.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:57.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.248023+0000", 2021-01-15T19:08:57.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011083061999999999 2021-01-15T19:08:57.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.560 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:57.560 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:57.560 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:57.568 INFO:tasks.rados.rados.0.smithi001.stdout:254: finishing write tid 5 to smithi00140323-42 2021-01-15T19:08:57.568 INFO:tasks.rados.rados.0.smithi001.stdout:254: finishing write tid 6 to smithi00140323-42 2021-01-15T19:08:57.568 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 26 (ObjNum 117 snap 26 seq_num 117) dirty exists 2021-01-15T19:08:57.569 INFO:tasks.rados.rados.0.smithi001.stdout:254: left oid 42 (ObjNum 117 snap 26 seq_num 117) 2021-01-15T19:08:57.569 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:08:57.569 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.569 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.570 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.570 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.570 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.570 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.570 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.571 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:08:57.571 INFO:teuthology.orchestra.run.smithi001.stdout:moved 0 / 136 (0%) 2021-01-15T19:08:57.571 INFO:teuthology.orchestra.run.smithi001.stdout:avg 17 2021-01-15T19:08:57.572 INFO:teuthology.orchestra.run.smithi001.stdout:stddev 3.60555 -> 3.60555 (expected baseline 3.85681) 2021-01-15T19:08:57.572 INFO:teuthology.orchestra.run.smithi001.stdout:min osd.7 with 11 -> 11 pgs (0.647059 -> 0.647059 * mean) 2021-01-15T19:08:57.572 INFO:teuthology.orchestra.run.smithi001.stdout:max osd.3 with 22 -> 22 pgs (1.29412 -> 1.29412 * mean) 2021-01-15T19:08:57.572 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:08:57.572 INFO:teuthology.orchestra.run.smithi001.stdout:oload 1000 2021-01-15T19:08:57.573 INFO:teuthology.orchestra.run.smithi001.stdout:max_change 1 2021-01-15T19:08:57.573 INFO:teuthology.orchestra.run.smithi001.stdout:max_change_osds 4 2021-01-15T19:08:57.573 INFO:teuthology.orchestra.run.smithi001.stdout:average_utilization 0.0003 2021-01-15T19:08:57.573 INFO:teuthology.orchestra.run.smithi001.stdout:overload_utilization 0.0031 2021-01-15T19:08:57.579 INFO:teuthology.orchestra.run.smithi001.stderr:no change 2021-01-15T19:08:57.580 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:08:57.580 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:57.560+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:57.664 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:57.662+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:57.722 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:57.716+0000 7f83bd5aa700 -1 osd.3 pg_epoch: 69 pg[3.9s0( v 69'41 (0'0,69'41] local-lis/les=52/53 n=5 ec=21/21 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,2]p3(0) r=0 lpr=52 luod=65'39 crt=56'38 lcod 56'38 mlcod 56'38 active+clean+snaptrim trimq=[14~1] ps=[10~1,1c~1,20~2,23~1]] removing snap head 2021-01-15T19:08:57.739 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:57.739 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:57.739 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:57.739 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:57.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:08:57.793 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:57.792+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:57.864 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:08:57.863+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:08:57.874 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:57.874 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:57.875 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:57.875 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:57.875 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:57.884 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:08:57.968 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:57.967+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:58.051 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:58.052 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:58.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:58.052 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:58.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.052 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:08:58.053 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:58.053 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.762960610999997, 2021-01-15T19:08:58.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:08:58.053 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.053 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.054 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.055 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.055 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.055 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:08:58.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.056 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:58.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:08:58.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:08:58.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:08:58.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:08:58.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:08:58.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:08:58.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:08:58.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:08:58.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:08:58.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:08:58.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:08:58.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:08:58.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:58.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:08:58.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:08:58.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:08:58.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:08:58.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:08:58.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:08:58.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:08:58.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:08:58.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:08:58.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:08:58.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:08:58.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:08:58.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:08:58.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:08:58.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.073 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.074 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:58.074 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:58.075 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.831197844999998, 2021-01-15T19:08:58.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:08:58.075 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.075 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.075 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.076 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.076 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.076 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:08:58.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.076 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:58.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:08:58.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:08:58.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:08:58.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:08:58.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:08:58.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:08:58.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:08:58.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:08:58.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:08:58.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:08:58.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:08:58.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:08:58.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:58.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:08:58.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:08:58.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:08:58.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:08:58.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:08:58.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:08:58.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:08:58.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:08:58.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:08:58.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:08:58.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:08:58.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:08:58.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:08:58.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:08:58.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.093 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.095 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:58.095 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:58.096 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.828909803999998, 2021-01-15T19:08:58.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:08:58.096 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.096 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.097 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.097 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:08:58.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.098 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:58.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:08:58.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:08:58.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:08:58.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:08:58.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:08:58.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:08:58.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:08:58.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:08:58.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:08:58.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:08:58.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:08:58.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:08:58.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:08:58.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:08:58.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:08:58.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:08:58.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:08:58.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:08:58.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:08:58.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:08:58.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:08:58.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:08:58.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.134 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:58.134 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:58.134 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.826320179, 2021-01-15T19:08:58.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:08:58.134 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.135 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.136 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:08:58.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.136 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:58.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:08:58.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:08:58.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:08:58.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:08:58.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:08:58.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:08:58.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:08:58.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:08:58.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:08:58.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:08:58.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:08:58.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:08:58.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:08:58.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:08:58.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:08:58.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:08:58.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:08:58.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:08:58.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:08:58.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:08:58.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:08:58.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:08:58.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:335 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (56) in=7b] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:58.153 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:58.153 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.825512735, 2021-01-15T19:08:58.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086755649999999997, 2021-01-15T19:08:58.154 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.154 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.154 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.155 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.155 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 335 2021-01-15T19:08:58.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.155 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:58.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223483+0000", 2021-01-15T19:08:58.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2554e-05 2021-01-15T19:08:58.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223515+0000", 2021-01-15T19:08:58.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.776e-06 2021-01-15T19:08:58.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223522+0000", 2021-01-15T19:08:58.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2351000000000001e-05 2021-01-15T19:08:58.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223534+0000", 2021-01-15T19:08:58.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4670000000000001e-06 2021-01-15T19:08:58.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223541+0000", 2021-01-15T19:08:58.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028742450000000001 2021-01-15T19:08:58.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226415+0000", 2021-01-15T19:08:58.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2873000000000005e-05 2021-01-15T19:08:58.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226488+0000", 2021-01-15T19:08:58.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000392001 2021-01-15T19:08:58.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.226880+0000", 2021-01-15T19:08:58.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061445900000000003 2021-01-15T19:08:58.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227494+0000", 2021-01-15T19:08:58.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045987420000000003 2021-01-15T19:08:58.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232093+0000", 2021-01-15T19:08:58.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5097000000000004e-05 2021-01-15T19:08:58.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232158+0000", 2021-01-15T19:08:58.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086755649999999997 2021-01-15T19:08:58.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:58.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:58.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.825411900999999, 2021-01-15T19:08:58.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:08:58.173 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.173 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.174 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:08:58.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.174 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:58.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:08:58.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:08:58.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:08:58.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:08:58.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:08:58.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:08:58.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:08:58.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:08:58.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:08:58.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:08:58.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:08:58.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:08:58.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:58.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:08:58.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:08:58.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:08:58.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:08:58.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:08:58.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:08:58.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:08:58.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:08:58.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.189 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:58.189 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:58.190 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.856727994, 2021-01-15T19:08:58.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:08:58.190 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.190 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.192 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:08:58.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.192 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:58.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:08:58.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:08:58.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:08:58.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:08:58.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:08:58.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:08:58.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:08:58.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:08:58.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:08:58.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:08:58.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:08:58.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:08:58.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:58.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:08:58.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:08:58.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:08:58.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:08:58.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:08:58.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:08:58.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:08:58.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:08:58.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:08:58.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:08:58.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:08:58.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:08:58.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:08:58.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:08:58.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.209 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.210 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:58.210 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:58.210 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.856578256999999, 2021-01-15T19:08:58.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:08:58.210 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.211 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.211 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:08:58.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.212 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:58.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:08:58.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:08:58.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:08:58.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:08:58.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:08:58.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:08:58.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:08:58.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:08:58.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:08:58.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:08:58.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:08:58.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:08:58.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:58.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:08:58.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:08:58.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:08:58.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:08:58.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:58.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:08:58.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:08:58.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:08:58.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:08:58.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:58.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:08:58.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:08:58.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:08:58.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:08:58.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:08:58.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:08:58.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:08:58.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:08:58.237 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.238 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.238 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:08:58.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:58.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.856426359, 2021-01-15T19:08:58.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:08:58.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.252 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:08:58.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.252 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:58.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:08:58.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:08:58.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:08:58.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:08:58.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:08:58.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:08:58.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:08:58.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:08:58.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:08:58.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:08:58.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:08:58.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:08:58.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:58.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:08:58.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:08:58.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:08:58.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:08:58.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:08:58.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:08:58.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:08:58.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:08:58.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:08:58.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:08:58.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:08:58.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:08:58.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:08:58.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:08:58.272 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.273 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.273 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:08:58.274 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:58.274 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.856301046999999, 2021-01-15T19:08:58.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:08:58.274 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.275 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:08:58.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:58.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:08:58.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:08:58.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:08:58.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:08:58.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:08:58.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:08:58.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:08:58.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:08:58.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:08:58.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:08:58.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:08:58.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:08:58.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:08:58.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:08:58.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:08:58.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:08:58.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:08:58.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:58.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:08:58.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:08:58.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:08:58.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:08:58.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:58.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:08:58.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:08:58.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:08:58.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:08:58.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:08:58.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:08:58.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:08:58.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:08:58.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.293 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.294 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:08:58.294 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:58.294 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.835468471999999, 2021-01-15T19:08:58.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:08:58.295 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.295 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.295 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.296 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:08:58.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:58.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:08:58.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:08:58.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:08:58.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:08:58.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:08:58.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:08:58.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:08:58.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:08:58.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:08:58.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:08:58.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:08:58.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:08:58.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:08:58.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:08:58.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:08:58.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:08:58.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:08:58.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:08:58.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:08:58.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:08:58.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:08:58.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:08:58.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:08:58.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:58.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.82917514, 2021-01-15T19:08:58.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:08:58.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.311 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.311 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:08:58.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.312 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:58.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:08:58.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:08:58.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:08:58.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:08:58.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:08:58.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:08:58.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:08:58.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:08:58.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:08:58.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:08:58.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:08:58.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:08:58.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:58.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:08:58.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:08:58.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:08:58.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:08:58.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:08:58.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:08:58.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:08:58.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:08:58.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.331 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:08:58.332 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:58.332 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.867115627, 2021-01-15T19:08:58.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:08:58.333 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.333 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.335 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:08:58.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.336 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:58.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:08:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:08:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:08:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:08:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:08:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:08:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:08:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:08:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:08:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:08:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:08:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:08:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:58.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:08:58.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:08:58.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:08:58.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:08:58.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:08:58.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:08:58.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:08:58.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:08:58.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:08:58.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:08:58.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:08:58.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:08:58.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:08:58.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:08:58.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.352 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.353 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:58.353 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:58.354 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.804743646999999, 2021-01-15T19:08:58.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:08:58.354 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.354 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.354 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.355 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:08:58.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.355 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:58.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:08:58.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:08:58.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:08:58.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:08:58.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:08:58.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:08:58.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:08:58.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:08:58.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:08:58.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:08:58.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:08:58.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:08:58.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:08:58.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:08:58.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:08:58.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:08:58.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:08:58.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:08:58.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:08:58.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:08:58.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:08:58.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:08:58.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.368 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.369 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:58.369 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:58.369 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.801441145999998, 2021-01-15T19:08:58.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:08:58.370 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.370 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.370 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.371 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:08:58.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:58.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:08:58.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:08:58.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:08:58.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:08:58.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:08:58.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:08:58.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:08:58.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:08:58.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:08:58.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:08:58.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:08:58.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:08:58.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:08:58.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:08:58.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:08:58.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:08:58.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:08:58.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:08:58.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:08:58.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:08:58.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:08:58.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:08:58.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:58.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:58.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.800119280000001, 2021-01-15T19:08:58.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:08:58.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:08:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:58.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:08:58.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:08:58.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:08:58.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:08:58.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:08:58.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:08:58.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:08:58.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:08:58.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:08:58.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:08:58.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:08:58.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:08:58.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:08:58.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:08:58.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:08:58.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:08:58.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:08:58.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:08:58.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:08:58.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:08:58.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:08:58.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:08:58.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.400 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:08:58.401 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:58.402 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.799749027000001, 2021-01-15T19:08:58.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:08:58.403 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.403 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.403 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.404 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:08:58.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:58.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:08:58.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:08:58.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:08:58.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:08:58.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:08:58.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:08:58.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:08:58.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:08:58.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:08:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:08:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:08:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:08:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:08:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:08:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:08:58.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:08:58.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:08:58.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:08:58.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:08:58.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:08:58.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:08:58.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:08:58.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.417 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.418 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:08:58.419 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:58.419 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.799414634000001, 2021-01-15T19:08:58.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:08:58.419 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.419 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.420 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:08:58.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:58.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:08:58.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:08:58.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:08:58.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:08:58.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:08:58.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:08:58.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:08:58.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:08:58.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:08:58.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:08:58.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:08:58.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:08:58.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:58.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:08:58.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:08:58.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:08:58.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:08:58.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:08:58.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:08:58.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:08:58.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:08:58.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.434 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.435 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.3.13:2 3.0s0 3:0c0290e4:::smithi00140323-22:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:58.435 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:58.436 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.79806507, 2021-01-15T19:08:58.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082549809999999998, 2021-01-15T19:08:58.436 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.436 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:58.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250930+0000", 2021-01-15T19:08:58.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9704999999999997e-05 2021-01-15T19:08:58.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251010+0000", 2021-01-15T19:08:58.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5044e-05 2021-01-15T19:08:58.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251025+0000", 2021-01-15T19:08:58.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.844e-05 2021-01-15T19:08:58.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251043+0000", 2021-01-15T19:08:58.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1319999999999996e-06 2021-01-15T19:08:58.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251050+0000", 2021-01-15T19:08:58.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029025309999999999 2021-01-15T19:08:58.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253953+0000", 2021-01-15T19:08:58.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010406199999999999 2021-01-15T19:08:58.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254057+0000", 2021-01-15T19:08:58.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005128067 2021-01-15T19:08:58.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259185+0000", 2021-01-15T19:08:58.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082549809999999998 2021-01-15T19:08:58.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.451 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.453 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:08:58.453 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:08:58.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8581533749999997, 2021-01-15T19:08:58.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:08:58.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:58.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:08:58.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:08:58.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:08:58.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:08:58.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:08:58.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:08:58.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:08:58.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:08:58.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:08:58.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:08:58.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:08:58.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:08:58.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:08:58.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:08:58.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:08:58.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:08:58.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:08:58.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:08:58.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:08:58.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:08:58.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:08:58.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:08:58.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.474 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:58.475 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:58.476 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:58.480 INFO:tasks.rados.rados.0.smithi001.stdout:251: done (6 left) 2021-01-15T19:08:58.481 INFO:tasks.rados.rados.0.smithi001.stdout:254: done (5 left) 2021-01-15T19:08:58.481 INFO:tasks.rados.rados.0.smithi001.stdout:255: done (4 left) 2021-01-15T19:08:58.481 INFO:tasks.rados.rados.0.smithi001.stdout:258: done (3 left) 2021-01-15T19:08:58.482 INFO:tasks.rados.rados.0.smithi001.stdout:259: read oid 39 snap -1 2021-01-15T19:08:58.482 INFO:tasks.rados.rados.0.smithi001.stdout:259: expect (ObjNum 99 snap 16 seq_num 99) 2021-01-15T19:08:58.483 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 8 current snap is 27 2021-01-15T19:08:58.483 INFO:tasks.rados.rados.0.smithi001.stdout:260: seq_num 118 ranges {3563520=786432,4349952=704512,5054464=499712} 2021-01-15T19:08:58.483 INFO:tasks.rados.rados.0.smithi001.stdout:260: writing smithi00140323-8 from 3563520 to 4349952 tid 1 2021-01-15T19:08:58.484 INFO:tasks.rados.rados.0.smithi001.stdout:260: writing smithi00140323-8 from 4349952 to 5054464 tid 2 2021-01-15T19:08:58.484 INFO:tasks.rados.rados.0.smithi001.stdout:260: writing smithi00140323-8 from 5054464 to 5554176 tid 3 2021-01-15T19:08:58.485 INFO:tasks.rados.rados.0.smithi001.stdout:261: read oid 20 snap 26 2021-01-15T19:08:58.485 INFO:tasks.rados.rados.0.smithi001.stdout:261: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:08:58.485 INFO:tasks.rados.rados.0.smithi001.stdout:262: snap_remove snap 21 2021-01-15T19:08:58.486 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:08:58.487 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:08:58.488 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:08:58.476+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:08:58.578 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:08:58.576+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:08:58.652 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:58.654 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:58.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:08:58.678 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:58.677+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:58.779 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:58.778+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:58.829 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:58.829 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:58.829 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:58.829 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:58.830 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:58.838 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:08:58.879 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:58.879+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:58.964 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:58.965 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:58.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:58.966 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:58.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.967 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:193 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~524288] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:08:58.967 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:58.968 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.706196120999998, 2021-01-15T19:08:58.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855, 2021-01-15T19:08:58.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:58.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:58.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:58.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:58.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:58.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 193 2021-01-15T19:08:58.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:58.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:58.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:58.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:58.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:58.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255548+0000", 2021-01-15T19:08:58.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002671896 2021-01-15T19:08:58.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:58.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258220+0000", 2021-01-15T19:08:58.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4620000000000004e-06 2021-01-15T19:08:58.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:58.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258225+0000", 2021-01-15T19:08:58.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1330000000000005e-06 2021-01-15T19:08:58.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:58.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258235+0000", 2021-01-15T19:08:58.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:58.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:58.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258241+0000", 2021-01-15T19:08:58.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2228000000000002e-05 2021-01-15T19:08:58.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:58.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258293+0000", 2021-01-15T19:08:58.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116582 2021-01-15T19:08:58.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:58.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.258410+0000", 2021-01-15T19:08:58.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001110085 2021-01-15T19:08:58.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:58.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259520+0000", 2021-01-15T19:08:58.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009106856 2021-01-15T19:08:58.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:58.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268627+0000", 2021-01-15T19:08:58.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011660200000000001 2021-01-15T19:08:58.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:58.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268743+0000", 2021-01-15T19:08:58.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1472999999999996e-05 2021-01-15T19:08:58.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:58.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:58.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:58.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.268835+0000", 2021-01-15T19:08:59.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013286855 2021-01-15T19:08:59.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.000 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.001 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:196 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:08:59.002 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:59.002 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.702321308999998, 2021-01-15T19:08:59.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001, 2021-01-15T19:08:59.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.003 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.004 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 196 2021-01-15T19:08:59.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:59.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259423+0000", 2021-01-15T19:08:59.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0012000000000002e-05 2021-01-15T19:08:59.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259463+0000", 2021-01-15T19:08:59.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.973e-06 2021-01-15T19:08:59.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259470+0000", 2021-01-15T19:08:59.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5406000000000001e-05 2021-01-15T19:08:59.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259485+0000", 2021-01-15T19:08:59.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3079999999999998e-06 2021-01-15T19:08:59.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259491+0000", 2021-01-15T19:08:59.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095828909999999996 2021-01-15T19:08:59.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269073+0000", 2021-01-15T19:08:59.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0439000000000002e-05 2021-01-15T19:08:59.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:59.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.269134+0000", 2021-01-15T19:08:59.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018984799999999999 2021-01-15T19:08:59.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271032+0000", 2021-01-15T19:08:59.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1051000000000006e-05 2021-01-15T19:08:59.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271123+0000", 2021-01-15T19:08:59.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074483999999999998 2021-01-15T19:08:59.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.271868+0000", 2021-01-15T19:08:59.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012445400000000001 2021-01-15T19:08:59.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.022 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:240 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~614400] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:59.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:59.024 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.677962276000002, 2021-01-15T19:08:59.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999, 2021-01-15T19:08:59.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.025 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 240 2021-01-15T19:08:59.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.025 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:59.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.283782+0000", 2021-01-15T19:08:59.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086219699999999999 2021-01-15T19:08:59.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284644+0000", 2021-01-15T19:08:59.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0584000000000001e-05 2021-01-15T19:08:59.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284655+0000", 2021-01-15T19:08:59.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6158999999999999e-05 2021-01-15T19:08:59.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284671+0000", 2021-01-15T19:08:59.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8340000000000005e-06 2021-01-15T19:08:59.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284680+0000", 2021-01-15T19:08:59.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7915000000000002e-05 2021-01-15T19:08:59.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284758+0000", 2021-01-15T19:08:59.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.323e-05 2021-01-15T19:08:59.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.284851+0000", 2021-01-15T19:08:59.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00082421600000000001 2021-01-15T19:08:59.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285675+0000", 2021-01-15T19:08:59.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010661781 2021-01-15T19:08:59.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296337+0000", 2021-01-15T19:08:59.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110252 2021-01-15T19:08:59.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296447+0000", 2021-01-15T19:08:59.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8554999999999994e-05 2021-01-15T19:08:59.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.296546+0000", 2021-01-15T19:08:59.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012763722999999999 2021-01-15T19:08:59.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.039 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.040 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:241 3.4s0 3:2d79b238:::smithi00140323-40:head [append 0~401408] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:59.040 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:59.041 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.675783072000002, 2021-01-15T19:08:59.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405, 2021-01-15T19:08:59.041 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.041 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.042 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 241 2021-01-15T19:08:59.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.042 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:59.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.285961+0000", 2021-01-15T19:08:59.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067619299999999995 2021-01-15T19:08:59.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286637+0000", 2021-01-15T19:08:59.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6490000000000008e-06 2021-01-15T19:08:59.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286645+0000", 2021-01-15T19:08:59.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4398e-05 2021-01-15T19:08:59.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286659+0000", 2021-01-15T19:08:59.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4090000000000003e-06 2021-01-15T19:08:59.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286668+0000", 2021-01-15T19:08:59.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001831538 2021-01-15T19:08:59.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288499+0000", 2021-01-15T19:08:59.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136229 2021-01-15T19:08:59.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.288636+0000", 2021-01-15T19:08:59.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061224300000000005 2021-01-15T19:08:59.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.289248+0000", 2021-01-15T19:08:59.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091558969999999996 2021-01-15T19:08:59.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298404+0000", 2021-01-15T19:08:59.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7905000000000006e-05 2021-01-15T19:08:59.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298492+0000", 2021-01-15T19:08:59.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6944e-05 2021-01-15T19:08:59.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298579+0000", 2021-01-15T19:08:59.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012617405 2021-01-15T19:08:59.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.055 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.055 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.055 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:242 3.4s0 3:2d79b238:::smithi00140323-40:head [setxattr _header (56) in=7b] snapc c=[c,7,3] ondisk+write+known_if_redirected e36)", 2021-01-15T19:08:59.056 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:59.056 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.675028707999999, 2021-01-15T19:08:59.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585, 2021-01-15T19:08:59.056 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.056 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.057 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 242 2021-01-15T19:08:59.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.058 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:59.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286715+0000", 2021-01-15T19:08:59.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.215e-05 2021-01-15T19:08:59.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286768+0000", 2021-01-15T19:08:59.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7579999999999992e-06 2021-01-15T19:08:59.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286775+0000", 2021-01-15T19:08:59.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1936999999999999e-05 2021-01-15T19:08:59.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286787+0000", 2021-01-15T19:08:59.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9110000000000001e-06 2021-01-15T19:08:59.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286792+0000", 2021-01-15T19:08:59.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036308260000000002 2021-01-15T19:08:59.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290423+0000", 2021-01-15T19:08:59.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000125501 2021-01-15T19:08:59.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290549+0000", 2021-01-15T19:08:59.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032349499999999998 2021-01-15T19:08:59.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.290872+0000", 2021-01-15T19:08:59.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077102209999999997 2021-01-15T19:08:59.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298582+0000", 2021-01-15T19:08:59.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0824e-05 2021-01-15T19:08:59.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298633+0000", 2021-01-15T19:08:59.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039596199999999997 2021-01-15T19:08:59.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299029+0000", 2021-01-15T19:08:59.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012313585 2021-01-15T19:08:59.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.070 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.071 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:243 3.4s0 3:2d79b238:::smithi00140323-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e36)", 2021-01-15T19:08:59.071 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:59.071 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.674908541999997, 2021-01-15T19:08:59.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453, 2021-01-15T19:08:59.071 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.072 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.072 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2021-01-15T19:08:59.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.073 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:59.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286836+0000", 2021-01-15T19:08:59.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0167999999999999e-05 2021-01-15T19:08:59.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286876+0000", 2021-01-15T19:08:59.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4409999999999998e-06 2021-01-15T19:08:59.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286882+0000", 2021-01-15T19:08:59.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2041e-05 2021-01-15T19:08:59.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286894+0000", 2021-01-15T19:08:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6410000000000001e-06 2021-01-15T19:08:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286899+0000", 2021-01-15T19:08:59.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046499439999999996 2021-01-15T19:08:59.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291549+0000", 2021-01-15T19:08:59.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104929 2021-01-15T19:08:59.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:59.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291654+0000", 2021-01-15T19:08:59.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079154610000000004 2021-01-15T19:08:59.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299569+0000", 2021-01-15T19:08:59.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112284 2021-01-15T19:08:59.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299682+0000", 2021-01-15T19:08:59.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093154399999999997 2021-01-15T19:08:59.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.300613+0000", 2021-01-15T19:08:59.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013777453 2021-01-15T19:08:59.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.085 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.086 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:318 3.1s0 3:8b30e322:::smithi00140323-24:head [copy-from ver 8] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:08:59.087 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:59.087 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.765707944999999, 2021-01-15T19:08:59.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729, 2021-01-15T19:08:59.087 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.087 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.087 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.088 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.088 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 318 2021-01-15T19:08:59.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.088 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:59.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196036+0000", 2021-01-15T19:08:59.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5969000000000005e-05 2021-01-15T19:08:59.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196102+0000", 2021-01-15T19:08:59.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6286e-05 2021-01-15T19:08:59.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196119+0000", 2021-01-15T19:08:59.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0645e-05 2021-01-15T19:08:59.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196139+0000", 2021-01-15T19:08:59.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1576000000000001e-05 2021-01-15T19:08:59.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196151+0000", 2021-01-15T19:08:59.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017478200000000001 2021-01-15T19:08:59.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196326+0000", 2021-01-15T19:08:59.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019917200000000001 2021-01-15T19:08:59.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196525+0000", 2021-01-15T19:08:59.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099001019999999992 2021-01-15T19:08:59.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.206425+0000", 2021-01-15T19:08:59.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043897240000000002 2021-01-15T19:08:59.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.210815+0000", 2021-01-15T19:08:59.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023466619999999998 2021-01-15T19:08:59.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213161+0000", 2021-01-15T19:08:59.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108811 2021-01-15T19:08:59.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.213270+0000", 2021-01-15T19:08:59.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017233729 2021-01-15T19:08:59.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:319 3.1s0 3:8b30e322:::smithi00140323-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:08:59.102 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:59.102 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.765545027000002, 2021-01-15T19:08:59.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001, 2021-01-15T19:08:59.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.103 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.103 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.103 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.103 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 319 2021-01-15T19:08:59.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.104 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:59.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196199+0000", 2021-01-15T19:08:59.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6667e-05 2021-01-15T19:08:59.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196236+0000", 2021-01-15T19:08:59.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4290000000000004e-06 2021-01-15T19:08:59.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196243+0000", 2021-01-15T19:08:59.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4992e-05 2021-01-15T19:08:59.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196258+0000", 2021-01-15T19:08:59.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6110000000000001e-06 2021-01-15T19:08:59.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196264+0000", 2021-01-15T19:08:59.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046740399999999999 2021-01-15T19:08:59.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196731+0000", 2021-01-15T19:08:59.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1105000000000004e-05 2021-01-15T19:08:59.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:08:59.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.196812+0000", 2021-01-15T19:08:59.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011315451000000001 2021-01-15T19:08:59.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208128+0000", 2021-01-15T19:08:59.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010873 2021-01-15T19:08:59.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:59.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.208237+0000", 2021-01-15T19:08:59.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058576419999999997 2021-01-15T19:08:59.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214094+0000", 2021-01-15T19:08:59.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119562 2021-01-15T19:08:59.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214214+0000", 2021-01-15T19:08:59.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012588000000000001 2021-01-15T19:08:59.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.214340+0000", 2021-01-15T19:08:59.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018140473000000001 2021-01-15T19:08:59.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.118 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.119 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:494 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:59.119 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:59.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.778531588, 2021-01-15T19:08:59.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1107716759999999, 2021-01-15T19:08:59.120 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.120 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.121 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 494 2021-01-15T19:08:59.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.121 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:59.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183213+0000", 2021-01-15T19:08:59.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.092e-05 2021-01-15T19:08:59.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183264+0000", 2021-01-15T19:08:59.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2300000000000008e-06 2021-01-15T19:08:59.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183272+0000", 2021-01-15T19:08:59.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6365e-05 2021-01-15T19:08:59.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183288+0000", 2021-01-15T19:08:59.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3640000000000001e-06 2021-01-15T19:08:59.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.183295+0000", 2021-01-15T19:08:59.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020715730000000002 2021-01-15T19:08:59.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185367+0000", 2021-01-15T19:08:59.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.627e-06 2021-01-15T19:08:59.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:59.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185370+0000", 2021-01-15T19:08:59.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0069268849999999 2021-01-15T19:08:59.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.192297+0000", 2021-01-15T19:08:59.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036647600000000002 2021-01-15T19:08:59.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.192663+0000", 2021-01-15T19:08:59.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.095695180000000005 2021-01-15T19:08:59.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.288358+0000", 2021-01-15T19:08:59.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112618 2021-01-15T19:08:59.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.288471+0000", 2021-01-15T19:08:59.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055134379999999998 2021-01-15T19:08:59.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.293984+0000", 2021-01-15T19:08:59.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1107716759999999 2021-01-15T19:08:59.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.134 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.135 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:495 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:59.135 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:59.135 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.77684788, 2021-01-15T19:08:59.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111341001, 2021-01-15T19:08:59.136 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.136 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.136 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.137 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 495 2021-01-15T19:08:59.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.137 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:59.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184896+0000", 2021-01-15T19:08:59.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4907999999999997e-05 2021-01-15T19:08:59.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184951+0000", 2021-01-15T19:08:59.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9130000000000003e-06 2021-01-15T19:08:59.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184959+0000", 2021-01-15T19:08:59.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4218e-05 2021-01-15T19:08:59.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184973+0000", 2021-01-15T19:08:59.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.212e-06 2021-01-15T19:08:59.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.184980+0000", 2021-01-15T19:08:59.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000418804 2021-01-15T19:08:59.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185398+0000", 2021-01-15T19:08:59.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2021-01-15T19:08:59.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:59.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185399+0000", 2021-01-15T19:08:59.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.007701593 2021-01-15T19:08:59.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193101+0000", 2021-01-15T19:08:59.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010404 2021-01-15T19:08:59.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193205+0000", 2021-01-15T19:08:59.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099598102999999993 2021-01-15T19:08:59.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.292803+0000", 2021-01-15T19:08:59.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122609 2021-01-15T19:08:59.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.292925+0000", 2021-01-15T19:08:59.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033118269999999998 2021-01-15T19:08:59.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.296237+0000", 2021-01-15T19:08:59.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111341001 2021-01-15T19:08:59.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.150 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:496 3.12s0 3:4a475518:::smithi00140323-15:19 [read 0~1097728,checksum] snapc 0=[] ondisk+read+known_if_redirected e55)", 2021-01-15T19:08:59.151 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:59.152 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.776723423, 2021-01-15T19:08:59.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1128853430000001, 2021-01-15T19:08:59.152 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.152 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.152 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.153 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 496 2021-01-15T19:08:59.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.153 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:59.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185021+0000", 2021-01-15T19:08:59.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5720000000000002e-05 2021-01-15T19:08:59.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185076+0000", 2021-01-15T19:08:59.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9819999999999997e-06 2021-01-15T19:08:59.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185085+0000", 2021-01-15T19:08:59.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3526000000000001e-05 2021-01-15T19:08:59.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185099+0000", 2021-01-15T19:08:59.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2099999999999998e-06 2021-01-15T19:08:59.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185105+0000", 2021-01-15T19:08:59.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036220000000000002 2021-01-15T19:08:59.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185467+0000", 2021-01-15T19:08:59.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:08:59.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:08:59.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.185469+0000", 2021-01-15T19:08:59.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.007797008 2021-01-15T19:08:59.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193266+0000", 2021-01-15T19:08:59.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102037 2021-01-15T19:08:59.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.193368+0000", 2021-01-15T19:08:59.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10068078 2021-01-15T19:08:59.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.294049+0000", 2021-01-15T19:08:59.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010173 2021-01-15T19:08:59.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.294151+0000", 2021-01-15T19:08:59.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037554530000000002 2021-01-15T19:08:59.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.297906+0000", 2021-01-15T19:08:59.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1128853430000001 2021-01-15T19:08:59.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.167 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.167 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.167 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:507 3.1s0 3:80f33866:::smithi00140323-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:08:59.168 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:59.168 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.729335645000001, 2021-01-15T19:08:59.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013129708, 2021-01-15T19:08:59.168 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.168 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.169 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.169 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 507 2021-01-15T19:08:59.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.170 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:59.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232409+0000", 2021-01-15T19:08:59.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7067000000000003e-05 2021-01-15T19:08:59.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232446+0000", 2021-01-15T19:08:59.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7929999999999992e-06 2021-01-15T19:08:59.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232455+0000", 2021-01-15T19:08:59.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1325000000000001e-05 2021-01-15T19:08:59.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232467+0000", 2021-01-15T19:08:59.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6519999999999997e-06 2021-01-15T19:08:59.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232471+0000", 2021-01-15T19:08:59.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006329187 2021-01-15T19:08:59.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.238801+0000", 2021-01-15T19:08:59.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010693569999999999 2021-01-15T19:08:59.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:59.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.239870+0000", 2021-01-15T19:08:59.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011900960000000001 2021-01-15T19:08:59.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.241060+0000", 2021-01-15T19:08:59.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012403300000000001 2021-01-15T19:08:59.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.241184+0000", 2021-01-15T19:08:59.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043541980000000001 2021-01-15T19:08:59.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.245538+0000", 2021-01-15T19:08:59.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013129708 2021-01-15T19:08:59.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.181 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.181 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.181 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:572 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~565248] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:59.182 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:59.182 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.710765629999999, 2021-01-15T19:08:59.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.058327720999999999, 2021-01-15T19:08:59.182 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.182 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.183 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 572 2021-01-15T19:08:59.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:59.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.250979+0000", 2021-01-15T19:08:59.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010807060000000001 2021-01-15T19:08:59.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252059+0000", 2021-01-15T19:08:59.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.392e-06 2021-01-15T19:08:59.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252066+0000", 2021-01-15T19:08:59.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1586999999999999e-05 2021-01-15T19:08:59.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252077+0000", 2021-01-15T19:08:59.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8189999999999999e-06 2021-01-15T19:08:59.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252082+0000", 2021-01-15T19:08:59.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4982000000000003e-05 2021-01-15T19:08:59.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252137+0000", 2021-01-15T19:08:59.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000471743 2021-01-15T19:08:59.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.252609+0000", 2021-01-15T19:08:59.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045366254000000002 2021-01-15T19:08:59.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.297975+0000", 2021-01-15T19:08:59.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106475 2021-01-15T19:08:59.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.298082+0000", 2021-01-15T19:08:59.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086370999999999996 2021-01-15T19:08:59.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.298945+0000", 2021-01-15T19:08:59.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068992059999999997 2021-01-15T19:08:59.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305844+0000", 2021-01-15T19:08:59.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032253609999999999 2021-01-15T19:08:59.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309070+0000", 2021-01-15T19:08:59.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236486 2021-01-15T19:08:59.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309306+0000", 2021-01-15T19:08:59.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.058327720999999999 2021-01-15T19:08:59.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.199 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.200 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:573 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~507904] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:59.200 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:59.200 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.706962668999999, 2021-01-15T19:08:59.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.057854161000000001, 2021-01-15T19:08:59.201 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.201 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.202 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 573 2021-01-15T19:08:59.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.202 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:59.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.254782+0000", 2021-01-15T19:08:59.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050420300000000005 2021-01-15T19:08:59.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255286+0000", 2021-01-15T19:08:59.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-06 2021-01-15T19:08:59.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255294+0000", 2021-01-15T19:08:59.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6209999999999999e-05 2021-01-15T19:08:59.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255310+0000", 2021-01-15T19:08:59.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2570000000000001e-06 2021-01-15T19:08:59.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255317+0000", 2021-01-15T19:08:59.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5633000000000002e-05 2021-01-15T19:08:59.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255363+0000", 2021-01-15T19:08:59.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7139999999999997e-05 2021-01-15T19:08:59.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.255440+0000", 2021-01-15T19:08:59.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045807546999999997 2021-01-15T19:08:59.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.301248+0000", 2021-01-15T19:08:59.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4977999999999998e-05 2021-01-15T19:08:59.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.301343+0000", 2021-01-15T19:08:59.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066354300000000005 2021-01-15T19:08:59.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.302006+0000", 2021-01-15T19:08:59.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056809900000000003 2021-01-15T19:08:59.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307687+0000", 2021-01-15T19:08:59.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045007709999999998 2021-01-15T19:08:59.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312188+0000", 2021-01-15T19:08:59.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044770900000000002 2021-01-15T19:08:59.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312636+0000", 2021-01-15T19:08:59.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.057854161000000001 2021-01-15T19:08:59.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.220 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.220 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.221 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:574 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~507904] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:59.221 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:59.221 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.703181624000001, 2021-01-15T19:08:59.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055246712000000003, 2021-01-15T19:08:59.222 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.222 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.223 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.223 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 574 2021-01-15T19:08:59.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.223 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:59.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258563+0000", 2021-01-15T19:08:59.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000591451 2021-01-15T19:08:59.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259154+0000", 2021-01-15T19:08:59.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9889999999999998e-06 2021-01-15T19:08:59.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259163+0000", 2021-01-15T19:08:59.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4970999999999999e-05 2021-01-15T19:08:59.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259178+0000", 2021-01-15T19:08:59.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.798e-06 2021-01-15T19:08:59.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.259185+0000", 2021-01-15T19:08:59.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022518479999999999 2021-01-15T19:08:59.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261437+0000", 2021-01-15T19:08:59.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107135 2021-01-15T19:08:59.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261544+0000", 2021-01-15T19:08:59.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041508633000000003 2021-01-15T19:08:59.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.303052+0000", 2021-01-15T19:08:59.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136876 2021-01-15T19:08:59.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.303189+0000", 2021-01-15T19:08:59.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000916328 2021-01-15T19:08:59.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.304106+0000", 2021-01-15T19:08:59.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052131690000000001 2021-01-15T19:08:59.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.309319+0000", 2021-01-15T19:08:59.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041306750000000003 2021-01-15T19:08:59.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313449+0000", 2021-01-15T19:08:59.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035983899999999998 2021-01-15T19:08:59.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313809+0000", 2021-01-15T19:08:59.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.055246712000000003 2021-01-15T19:08:59.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.240 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.241 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:575 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~188416] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:59.241 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:59.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.701615367, 2021-01-15T19:08:59.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054232529000000002, 2021-01-15T19:08:59.242 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.242 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.243 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 575 2021-01-15T19:08:59.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.243 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:59.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260129+0000", 2021-01-15T19:08:59.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000227791 2021-01-15T19:08:59.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260357+0000", 2021-01-15T19:08:59.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2290000000000003e-06 2021-01-15T19:08:59.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260365+0000", 2021-01-15T19:08:59.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6650000000000002e-05 2021-01-15T19:08:59.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260382+0000", 2021-01-15T19:08:59.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1879999999999996e-06 2021-01-15T19:08:59.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260389+0000", 2021-01-15T19:08:59.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001199133 2021-01-15T19:08:59.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261588+0000", 2021-01-15T19:08:59.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4250999999999999e-05 2021-01-15T19:08:59.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261642+0000", 2021-01-15T19:08:59.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044223854999999999 2021-01-15T19:08:59.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305866+0000", 2021-01-15T19:08:59.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8614000000000001e-05 2021-01-15T19:08:59.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.305965+0000", 2021-01-15T19:08:59.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060702 2021-01-15T19:08:59.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.306572+0000", 2021-01-15T19:08:59.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006392488 2021-01-15T19:08:59.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.312964+0000", 2021-01-15T19:08:59.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010513219999999999 2021-01-15T19:08:59.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314015+0000", 2021-01-15T19:08:59.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034598799999999998 2021-01-15T19:08:59.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314361+0000", 2021-01-15T19:08:59.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054232529000000002 2021-01-15T19:08:59.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.260 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.261 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:576 3.12s0 3:4e4266df:::smithi00140323-37:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:08:59.261 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:59.261 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.701310935, 2021-01-15T19:08:59.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054314145000000001, 2021-01-15T19:08:59.262 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.262 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.262 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.263 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 576 2021-01-15T19:08:59.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.263 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:59.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260433+0000", 2021-01-15T19:08:59.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5556000000000002e-05 2021-01-15T19:08:59.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260479+0000", 2021-01-15T19:08:59.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0480000000000004e-06 2021-01-15T19:08:59.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260485+0000", 2021-01-15T19:08:59.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3729999999999993e-06 2021-01-15T19:08:59.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260493+0000", 2021-01-15T19:08:59.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9958000000000001e-05 2021-01-15T19:08:59.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260513+0000", 2021-01-15T19:08:59.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00117347 2021-01-15T19:08:59.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261687+0000", 2021-01-15T19:08:59.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5226000000000003e-05 2021-01-15T19:08:59.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261742+0000", 2021-01-15T19:08:59.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045966993999999997 2021-01-15T19:08:59.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307709+0000", 2021-01-15T19:08:59.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3752999999999995e-05 2021-01-15T19:08:59.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.307803+0000", 2021-01-15T19:08:59.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057255800000000003 2021-01-15T19:08:59.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308375+0000", 2021-01-15T19:08:59.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046299920000000003 2021-01-15T19:08:59.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.313005+0000", 2021-01-15T19:08:59.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001483086 2021-01-15T19:08:59.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314488+0000", 2021-01-15T19:08:59.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025913100000000001 2021-01-15T19:08:59.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.314747+0000", 2021-01-15T19:08:59.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.054314145000000001 2021-01-15T19:08:59.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.280 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:577 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:08:59.281 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:59.281 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.701189999, 2021-01-15T19:08:59.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056902372999999999, 2021-01-15T19:08:59.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.284 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 577 2021-01-15T19:08:59.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.284 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:59.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260554+0000", 2021-01-15T19:08:59.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0988000000000001e-05 2021-01-15T19:08:59.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260595+0000", 2021-01-15T19:08:59.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5379999999999999e-06 2021-01-15T19:08:59.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260602+0000", 2021-01-15T19:08:59.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2503e-05 2021-01-15T19:08:59.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260614+0000", 2021-01-15T19:08:59.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7730000000000002e-06 2021-01-15T19:08:59.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.260621+0000", 2021-01-15T19:08:59.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011648749999999999 2021-01-15T19:08:59.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261786+0000", 2021-01-15T19:08:59.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111148 2021-01-15T19:08:59.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for missing object", 2021-01-15T19:08:59.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.261897+0000", 2021-01-15T19:08:59.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.046774885000000002 2021-01-15T19:08:59.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308672+0000", 2021-01-15T19:08:59.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014140499999999999 2021-01-15T19:08:59.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:08:59.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.308813+0000", 2021-01-15T19:08:59.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063629660000000003 2021-01-15T19:08:59.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.315176+0000", 2021-01-15T19:08:59.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014743699999999999 2021-01-15T19:08:59.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.315324+0000", 2021-01-15T19:08:59.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021328549999999999 2021-01-15T19:08:59.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.317457+0000", 2021-01-15T19:08:59.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056902372999999999 2021-01-15T19:08:59.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:644 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~696320] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:08:59.301 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.216658+0000", 2021-01-15T19:08:59.301 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7450864289999997, 2021-01-15T19:08:59.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013177065999999999, 2021-01-15T19:08:59.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.302 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 644 2021-01-15T19:08:59.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.303 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216658+0000", 2021-01-15T19:08:59.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216658+0000", 2021-01-15T19:08:59.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015732630000000001 2021-01-15T19:08:59.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.218231+0000", 2021-01-15T19:08:59.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3769999999999996e-06 2021-01-15T19:08:59.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.218239+0000", 2021-01-15T19:08:59.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4548e-05 2021-01-15T19:08:59.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.218254+0000", 2021-01-15T19:08:59.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2870000000000004e-06 2021-01-15T19:08:59.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.218259+0000", 2021-01-15T19:08:59.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1427999999999997e-05 2021-01-15T19:08:59.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.218301+0000", 2021-01-15T19:08:59.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3436000000000006e-05 2021-01-15T19:08:59.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.218374+0000", 2021-01-15T19:08:59.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077393000000000004 2021-01-15T19:08:59.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.219148+0000", 2021-01-15T19:08:59.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079053939999999996 2021-01-15T19:08:59.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.227053+0000", 2021-01-15T19:08:59.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002680984 2021-01-15T19:08:59.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.229734+0000", 2021-01-15T19:08:59.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100419 2021-01-15T19:08:59.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.229835+0000", 2021-01-15T19:08:59.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013177065999999999 2021-01-15T19:08:59.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.316 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:645 3.1s0 3:80f33866:::smithi00140323-14:head [append 0~434176] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:08:59.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.219592+0000", 2021-01-15T19:08:59.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7421522019999998, 2021-01-15T19:08:59.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012821031, 2021-01-15T19:08:59.317 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.319 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 645 2021-01-15T19:08:59.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.319 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.219592+0000", 2021-01-15T19:08:59.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.219592+0000", 2021-01-15T19:08:59.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065805099999999999 2021-01-15T19:08:59.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.220250+0000", 2021-01-15T19:08:59.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2609999999999999e-06 2021-01-15T19:08:59.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.220259+0000", 2021-01-15T19:08:59.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4358999999999999e-05 2021-01-15T19:08:59.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.220274+0000", 2021-01-15T19:08:59.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.905e-06 2021-01-15T19:08:59.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.220281+0000", 2021-01-15T19:08:59.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075868699999999997 2021-01-15T19:08:59.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221039+0000", 2021-01-15T19:08:59.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012280900000000001 2021-01-15T19:08:59.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221162+0000", 2021-01-15T19:08:59.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076917700000000001 2021-01-15T19:08:59.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221931+0000", 2021-01-15T19:08:59.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079712600000000008 2021-01-15T19:08:59.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.229903+0000", 2021-01-15T19:08:59.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002432465 2021-01-15T19:08:59.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:08:59.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.232335+0000", 2021-01-15T19:08:59.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8057000000000002e-05 2021-01-15T19:08:59.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.232413+0000", 2021-01-15T19:08:59.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012821031 2021-01-15T19:08:59.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.334 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:59.337 INFO:tasks.rados.rados.0.smithi001.stdout:262: done (6 left) 2021-01-15T19:08:59.337 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 27 2021-01-15T19:08:59.337 INFO:tasks.rados.rados.0.smithi001.stdout:263: seq_num 119 ranges {1900544=802816,2703360=589824,3293184=409600,3702784=204800} 2021-01-15T19:08:59.337 INFO:tasks.rados.rados.0.smithi001.stdout:263: writing smithi00140323-14 from 1900544 to 2703360 tid 1 2021-01-15T19:08:59.337 INFO:tasks.rados.rados.0.smithi001.stdout:263: writing smithi00140323-14 from 2703360 to 3293184 tid 2 2021-01-15T19:08:59.338 INFO:tasks.rados.rados.0.smithi001.stdout:263: writing smithi00140323-14 from 3293184 to 3702784 tid 3 2021-01-15T19:08:59.338 INFO:tasks.rados.rados.0.smithi001.stdout:263: writing smithi00140323-14 from 3702784 to 3907584 tid 4 2021-01-15T19:08:59.338 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 27 2021-01-15T19:08:59.338 INFO:tasks.rados.rados.0.smithi001.stdout:264: seq_num 120 ranges {4300800=417792,4718592=475136,5193728=622592,5816320=507904} 2021-01-15T19:08:59.338 INFO:tasks.rados.rados.0.smithi001.stdout:264: writing smithi00140323-13 from 4300800 to 4718592 tid 1 2021-01-15T19:08:59.339 INFO:tasks.rados.rados.0.smithi001.stdout:264: writing smithi00140323-13 from 4718592 to 5193728 tid 2 2021-01-15T19:08:59.339 INFO:tasks.rados.rados.0.smithi001.stdout:264: writing smithi00140323-13 from 5193728 to 5816320 tid 3 2021-01-15T19:08:59.339 INFO:tasks.rados.rados.0.smithi001.stdout:264: writing smithi00140323-13 from 5816320 to 6324224 tid 4 2021-01-15T19:08:59.339 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 7 current snap is 27 2021-01-15T19:08:59.340 INFO:tasks.rados.rados.0.smithi001.stdout:265: seq_num 121 ranges {0=434176,434176=753664,1187840=425984,1613824=425984} 2021-01-15T19:08:59.340 INFO:tasks.rados.rados.0.smithi001.stdout:265: writing smithi00140323-7 from 0 to 434176 tid 1 2021-01-15T19:08:59.340 INFO:tasks.rados.rados.0.smithi001.stdout:265: writing smithi00140323-7 from 434176 to 1187840 tid 2 2021-01-15T19:08:59.340 INFO:tasks.rados.rados.0.smithi001.stdout:265: writing smithi00140323-7 from 1187840 to 1613824 tid 3 2021-01-15T19:08:59.340 INFO:tasks.rados.rados.0.smithi001.stdout:265: writing smithi00140323-7 from 1613824 to 2039808 tid 4 2021-01-15T19:08:59.341 INFO:tasks.rados.rados.0.smithi001.stdout:266: copy_from oid 33 from oid 11 current snap is 27 2021-01-15T19:08:59.341 INFO:tasks.rados.rados.0.smithi001.stdout:261: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:08:59.341 INFO:tasks.rados.rados.0.smithi001.stdout:260: finishing write tid 1 to smithi00140323-8 2021-01-15T19:08:59.341 INFO:tasks.rados.rados.0.smithi001.stdout:260: finishing write tid 2 to smithi00140323-8 2021-01-15T19:08:59.341 INFO:tasks.rados.rados.0.smithi001.stdout:260: finishing write tid 3 to smithi00140323-8 2021-01-15T19:08:59.342 INFO:tasks.rados.rados.0.smithi001.stdout:260: finishing write tid 4 to smithi00140323-8 2021-01-15T19:08:59.342 INFO:tasks.rados.rados.0.smithi001.stdout:260: finishing write tid 5 to smithi00140323-8 2021-01-15T19:08:59.342 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 48 (ObjNum 118 snap 27 seq_num 118) dirty exists 2021-01-15T19:08:59.342 INFO:tasks.rados.rados.0.smithi001.stdout:260: left oid 8 (ObjNum 118 snap 27 seq_num 118) 2021-01-15T19:08:59.342 INFO:tasks.rados.rados.0.smithi001.stdout:259: expect (ObjNum 99 snap 16 seq_num 99) 2021-01-15T19:08:59.343 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:59.333+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:59.343 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:08:59.436 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:08:59.435+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:08:59.469 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:59.469 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:59.469 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:08:59.470 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:59.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:08:59.486 INFO:tasks.rados.rados.0.smithi001.stdout:263: finishing write tid 1 to smithi00140323-14 2021-01-15T19:08:59.486 INFO:tasks.rados.rados.0.smithi001.stdout:263: finishing write tid 2 to smithi00140323-14 2021-01-15T19:08:59.486 INFO:tasks.rados.rados.0.smithi001.stdout:263: finishing write tid 3 to smithi00140323-14 2021-01-15T19:08:59.487 INFO:tasks.rados.rados.0.smithi001.stdout:263: finishing write tid 4 to smithi00140323-14 2021-01-15T19:08:59.487 INFO:tasks.rados.rados.0.smithi001.stdout:263: finishing write tid 5 to smithi00140323-14 2021-01-15T19:08:59.487 INFO:tasks.rados.rados.0.smithi001.stdout:263: finishing write tid 6 to smithi00140323-14 2021-01-15T19:08:59.487 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 66 (ObjNum 119 snap 27 seq_num 119) dirty exists 2021-01-15T19:08:59.487 INFO:tasks.rados.rados.0.smithi001.stdout:263: left oid 14 (ObjNum 119 snap 27 seq_num 119) 2021-01-15T19:08:59.488 INFO:tasks.rados.rados.0.smithi001.stdout:264: finishing write tid 1 to smithi00140323-13 2021-01-15T19:08:59.488 INFO:tasks.rados.rados.0.smithi001.stdout:264: finishing write tid 2 to smithi00140323-13 2021-01-15T19:08:59.488 INFO:tasks.rados.rados.0.smithi001.stdout:264: finishing write tid 3 to smithi00140323-13 2021-01-15T19:08:59.488 INFO:tasks.rados.rados.0.smithi001.stdout:264: finishing write tid 4 to smithi00140323-13 2021-01-15T19:08:59.488 INFO:tasks.rados.rados.0.smithi001.stdout:265: finishing write tid 1 to smithi00140323-7 2021-01-15T19:08:59.489 INFO:tasks.rados.rados.0.smithi001.stdout:264: finishing write tid 5 to smithi00140323-13 2021-01-15T19:08:59.489 INFO:tasks.rados.rados.0.smithi001.stdout:264: finishing write tid 6 to smithi00140323-13 2021-01-15T19:08:59.489 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 89 (ObjNum 120 snap 27 seq_num 120) dirty exists 2021-01-15T19:08:59.489 INFO:tasks.rados.rados.0.smithi001.stdout:264: left oid 13 (ObjNum 120 snap 27 seq_num 120) 2021-01-15T19:08:59.489 INFO:tasks.rados.rados.0.smithi001.stdout:265: finishing write tid 2 to smithi00140323-7 2021-01-15T19:08:59.490 INFO:tasks.rados.rados.0.smithi001.stdout:265: finishing write tid 3 to smithi00140323-7 2021-01-15T19:08:59.490 INFO:tasks.rados.rados.0.smithi001.stdout:259: done (9 left) 2021-01-15T19:08:59.490 INFO:tasks.rados.rados.0.smithi001.stdout:260: done (8 left) 2021-01-15T19:08:59.490 INFO:tasks.rados.rados.0.smithi001.stdout:261: done (7 left) 2021-01-15T19:08:59.491 INFO:tasks.rados.rados.0.smithi001.stdout:263: done (6 left) 2021-01-15T19:08:59.491 INFO:tasks.rados.rados.0.smithi001.stdout:264: done (5 left) 2021-01-15T19:08:59.491 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 29 current snap is 27 2021-01-15T19:08:59.491 INFO:tasks.rados.rados.0.smithi001.stdout:267: seq_num 122 ranges {0=450560,450560=688128,1138688=630784,1769472=286720} 2021-01-15T19:08:59.492 INFO:tasks.rados.rados.0.smithi001.stdout:267: writing smithi00140323-29 from 0 to 450560 tid 1 2021-01-15T19:08:59.492 INFO:tasks.rados.rados.0.smithi001.stdout:267: writing smithi00140323-29 from 450560 to 1138688 tid 2 2021-01-15T19:08:59.496 INFO:tasks.rados.rados.0.smithi001.stdout:267: writing smithi00140323-29 from 1138688 to 1769472 tid 3 2021-01-15T19:08:59.498 INFO:tasks.rados.rados.0.smithi001.stdout:267: writing smithi00140323-29 from 1769472 to 2056192 tid 4 2021-01-15T19:08:59.498 INFO:tasks.rados.rados.0.smithi001.stdout:268: snap_create 2021-01-15T19:08:59.499 INFO:tasks.rados.rados.0.smithi001.stdout:265: finishing write tid 4 to smithi00140323-7 2021-01-15T19:08:59.499 INFO:tasks.rados.rados.0.smithi001.stdout:265: finishing write tid 5 to smithi00140323-7 2021-01-15T19:08:59.499 INFO:tasks.rados.rados.0.smithi001.stdout:265: finishing write tid 6 to smithi00140323-7 2021-01-15T19:08:59.499 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 94 (ObjNum 121 snap 27 seq_num 121) dirty exists 2021-01-15T19:08:59.500 INFO:tasks.rados.rados.0.smithi001.stdout:265: left oid 7 (ObjNum 121 snap 27 seq_num 121) 2021-01-15T19:08:59.500 INFO:tasks.rados.rados.0.smithi001.stdout:266: finishing copy_from to smithi00140323-33 2021-01-15T19:08:59.500 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 67 (ObjNum 102 snap 16 seq_num 102) dirty exists 2021-01-15T19:08:59.500 INFO:tasks.rados.rados.0.smithi001.stdout:266: finishing copy_from racing read to smithi00140323-33 2021-01-15T19:08:59.500 INFO:tasks.rados.rados.0.smithi001.stdout:267: finishing write tid 1 to smithi00140323-29 2021-01-15T19:08:59.501 INFO:tasks.rados.rados.0.smithi001.stdout:267: finishing write tid 2 to smithi00140323-29 2021-01-15T19:08:59.501 INFO:tasks.rados.rados.0.smithi001.stdout:267: finishing write tid 3 to smithi00140323-29 2021-01-15T19:08:59.502 INFO:tasks.rados.rados.0.smithi001.stdout:267: finishing write tid 4 to smithi00140323-29 2021-01-15T19:08:59.503 INFO:tasks.rados.rados.0.smithi001.stdout:267: finishing write tid 5 to smithi00140323-29 2021-01-15T19:08:59.504 INFO:tasks.rados.rados.0.smithi001.stdout:267: finishing write tid 6 to smithi00140323-29 2021-01-15T19:08:59.504 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 72 (ObjNum 122 snap 27 seq_num 122) dirty exists 2021-01-15T19:08:59.505 INFO:tasks.rados.rados.0.smithi001.stdout:267: left oid 29 (ObjNum 122 snap 27 seq_num 122) 2021-01-15T19:08:59.539 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:08:59.537+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:08:59.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:08:59.639 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:59.638+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:59.643 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:59.643 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:08:59.643 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:08:59.643 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:08:59.644 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:08:59.652 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:08:59.724 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:08:59.740 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:08:59.739+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:08:59.744 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:08:59.764 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:08:59.783 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:08:59.784 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:08:59.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:08:59.784 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:08:59.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.785 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 41/21 ECSubWrite(tid=43, reqid=client.4418.0:333, at_version=41'24, trim_to=0'0, roll_forward_to=39'22))", 2021-01-15T19:08:59.786 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:59.786 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.557173730999999, 2021-01-15T19:08:59.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001, 2021-01-15T19:08:59.787 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.787 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.788 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:59.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223599+0000", 2021-01-15T19:08:59.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0464999999999997e-05 2021-01-15T19:08:59.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223670+0000", 2021-01-15T19:08:59.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3445e-05 2021-01-15T19:08:59.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223693+0000", 2021-01-15T19:08:59.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.137e-05 2021-01-15T19:08:59.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223705+0000", 2021-01-15T19:08:59.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0939999999999996e-06 2021-01-15T19:08:59.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223711+0000", 2021-01-15T19:08:59.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4654999999999999e-05 2021-01-15T19:08:59.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223755+0000", 2021-01-15T19:08:59.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1819e-05 2021-01-15T19:08:59.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223767+0000", 2021-01-15T19:08:59.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076093979999999999 2021-01-15T19:08:59.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231376+0000", 2021-01-15T19:08:59.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4353999999999997e-05 2021-01-15T19:08:59.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231411+0000", 2021-01-15T19:08:59.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078116000000000001 2021-01-15T19:08:59.806 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.806 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.806 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=50'50, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:59.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:59.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.599434094999999, 2021-01-15T19:08:59.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996, 2021-01-15T19:08:59.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:59.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181339+0000", 2021-01-15T19:08:59.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9509999999999999e-05 2021-01-15T19:08:59.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181388+0000", 2021-01-15T19:08:59.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.141e-05 2021-01-15T19:08:59.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181410+0000", 2021-01-15T19:08:59.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5201000000000001e-05 2021-01-15T19:08:59.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181425+0000", 2021-01-15T19:08:59.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5440000000000004e-06 2021-01-15T19:08:59.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181432+0000", 2021-01-15T19:08:59.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029235099999999999 2021-01-15T19:08:59.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181724+0000", 2021-01-15T19:08:59.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1324e-05 2021-01-15T19:08:59.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181735+0000", 2021-01-15T19:08:59.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005959865 2021-01-15T19:08:59.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187695+0000", 2021-01-15T19:08:59.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4237999999999999e-05 2021-01-15T19:08:59.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187739+0000", 2021-01-15T19:08:59.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064004429999999996 2021-01-15T19:08:59.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 50/21 ECSubWrite(tid=73, reqid=osd.5.0:73, at_version=50'52, trim_to=0'0, roll_forward_to=49'48))", 2021-01-15T19:08:59.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:59.820 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.599020414000002, 2021-01-15T19:08:59.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003, 2021-01-15T19:08:59.820 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:59.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181753+0000", 2021-01-15T19:08:59.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7224000000000001e-05 2021-01-15T19:08:59.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181800+0000", 2021-01-15T19:08:59.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5681999999999999e-05 2021-01-15T19:08:59.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181825+0000", 2021-01-15T19:08:59.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5474000000000001e-05 2021-01-15T19:08:59.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181841+0000", 2021-01-15T19:08:59.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8179999999999993e-06 2021-01-15T19:08:59.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.181850+0000", 2021-01-15T19:08:59.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150703 2021-01-15T19:08:59.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182000+0000", 2021-01-15T19:08:59.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5359999999999998e-06 2021-01-15T19:08:59.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.182010+0000", 2021-01-15T19:08:59.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066268780000000001 2021-01-15T19:08:59.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188637+0000", 2021-01-15T19:08:59.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6959000000000002e-05 2021-01-15T19:08:59.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.188684+0000", 2021-01-15T19:08:59.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069312740000000003 2021-01-15T19:08:59.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.833 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.833 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.833 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=152, reqid=osd.4.0:152, at_version=50'30, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:59.834 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:59.834 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.596003760999999, 2021-01-15T19:08:59.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998, 2021-01-15T19:08:59.834 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.834 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.835 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:59.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184769+0000", 2021-01-15T19:08:59.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1204e-05 2021-01-15T19:08:59.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184820+0000", 2021-01-15T19:08:59.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1126000000000001e-05 2021-01-15T19:08:59.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184842+0000", 2021-01-15T19:08:59.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5197999999999999e-05 2021-01-15T19:08:59.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184857+0000", 2021-01-15T19:08:59.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.336e-06 2021-01-15T19:08:59.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184863+0000", 2021-01-15T19:08:59.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2200000000000002e-05 2021-01-15T19:08:59.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184915+0000", 2021-01-15T19:08:59.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999e-05 2021-01-15T19:08:59.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184927+0000", 2021-01-15T19:08:59.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065179089999999997 2021-01-15T19:08:59.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191445+0000", 2021-01-15T19:08:59.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2212999999999997e-05 2021-01-15T19:08:59.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.191487+0000", 2021-01-15T19:08:59.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067181849999999998 2021-01-15T19:08:59.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.846 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.847 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 50/21 ECSubWrite(tid=153, reqid=osd.4.0:153, at_version=50'32, trim_to=0'0, roll_forward_to=49'28))", 2021-01-15T19:08:59.848 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:59.848 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.595861412000001, 2021-01-15T19:08:59.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999, 2021-01-15T19:08:59.849 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.849 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:59.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184912+0000", 2021-01-15T19:08:59.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0540000000000001e-05 2021-01-15T19:08:59.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184962+0000", 2021-01-15T19:08:59.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1452999999999999e-05 2021-01-15T19:08:59.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184984+0000", 2021-01-15T19:08:59.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4532000000000001e-05 2021-01-15T19:08:59.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.184998+0000", 2021-01-15T19:08:59.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9069999999999998e-06 2021-01-15T19:08:59.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.185003+0000", 2021-01-15T19:08:59.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027679219999999999 2021-01-15T19:08:59.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187771+0000", 2021-01-15T19:08:59.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1089e-05 2021-01-15T19:08:59.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.187782+0000", 2021-01-15T19:08:59.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004853792 2021-01-15T19:08:59.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192636+0000", 2021-01-15T19:08:59.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0936000000000003e-05 2021-01-15T19:08:59.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.192667+0000", 2021-01-15T19:08:59.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077551709999999999 2021-01-15T19:08:59.866 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.866 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.866 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:08:59.867 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:59.868 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.596051699, 2021-01-15T19:08:59.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:08:59.868 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.868 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:59.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:08:59.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:08:59.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:08:59.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:08:59.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:08:59.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:08:59.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:08:59.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:08:59.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:08:59.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:08:59.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:08:59.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:08:59.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:08:59.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:08:59.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:08:59.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:08:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:08:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:08:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:439 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.516762344, 2021-01-15T19:08:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007, 2021-01-15T19:08:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 439 2021-01-15T19:08:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264011+0000", 2021-01-15T19:08:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6384000000000001e-05 2021-01-15T19:08:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264067+0000", 2021-01-15T19:08:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7919999999999998e-06 2021-01-15T19:08:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264076+0000", 2021-01-15T19:08:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6671e-05 2021-01-15T19:08:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264092+0000", 2021-01-15T19:08:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7889999999999995e-06 2021-01-15T19:08:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264100+0000", 2021-01-15T19:08:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3668999999999997e-05 2021-01-15T19:08:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264154+0000", 2021-01-15T19:08:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010867800000000001 2021-01-15T19:08:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264263+0000", 2021-01-15T19:08:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008062049 2021-01-15T19:08:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.272325+0000", 2021-01-15T19:08:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083140320000000007 2021-01-15T19:08:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:440 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.516624691000001, 2021-01-15T19:08:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695, 2021-01-15T19:08:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 440 2021-01-15T19:08:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264148+0000", 2021-01-15T19:08:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1341000000000002e-05 2021-01-15T19:08:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264200+0000", 2021-01-15T19:08:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8589999999999999e-06 2021-01-15T19:08:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264206+0000", 2021-01-15T19:08:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1953000000000001e-05 2021-01-15T19:08:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264218+0000", 2021-01-15T19:08:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.276e-06 2021-01-15T19:08:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264224+0000", 2021-01-15T19:08:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000377647 2021-01-15T19:08:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264601+0000", 2021-01-15T19:08:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8468999999999996e-05 2021-01-15T19:08:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264690+0000", 2021-01-15T19:08:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095141500000000007 2021-01-15T19:08:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274204+0000", 2021-01-15T19:08:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055695 2021-01-15T19:08:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:441 3.bs0 3:d7abb2d9:::smithi00140323-30:head [read 0~1892352,checksum] snapc 0=[] ondisk+read+known_if_redirected e51)", 2021-01-15T19:08:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.516504231999999, 2021-01-15T19:08:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001, 2021-01-15T19:08:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:08:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 441 2021-01-15T19:08:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264269+0000", 2021-01-15T19:08:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0819000000000001e-05 2021-01-15T19:08:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264340+0000", 2021-01-15T19:08:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3950000000000001e-06 2021-01-15T19:08:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264348+0000", 2021-01-15T19:08:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2213e-05 2021-01-15T19:08:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264360+0000", 2021-01-15T19:08:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6470000000000001e-06 2021-01-15T19:08:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264367+0000", 2021-01-15T19:08:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041851429999999997 2021-01-15T19:08:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268552+0000", 2021-01-15T19:08:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108403 2021-01-15T19:08:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:08:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.268660+0000", 2021-01-15T19:08:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006812706 2021-01-15T19:08:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275473+0000", 2021-01-15T19:08:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011204326000000001 2021-01-15T19:08:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.515094249000001, 2021-01-15T19:08:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:08:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:08:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:08:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:08:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:08:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:08:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:08:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:08:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:08:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:08:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:08:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:08:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:08:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:08:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:08:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:08:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:08:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:08:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:08:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.510682975000002, 2021-01-15T19:08:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:08:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:08:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:08:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:08:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:08:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:08:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:08:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:08:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:08:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:08:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:08:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:08:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:08:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:08:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:08:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:08:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:08:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:08:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:08:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=106, reqid=client.4418.0:436, at_version=51'42, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.507574830999999, 2021-01-15T19:08:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049, 2021-01-15T19:08:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273198+0000", 2021-01-15T19:08:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184005 2021-01-15T19:08:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273382+0000", 2021-01-15T19:08:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3705999999999999e-05 2021-01-15T19:08:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273416+0000", 2021-01-15T19:08:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1336999999999999e-05 2021-01-15T19:08:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273437+0000", 2021-01-15T19:08:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4830000000000005e-06 2021-01-15T19:08:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273447+0000", 2021-01-15T19:08:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001793602 2021-01-15T19:08:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275240+0000", 2021-01-15T19:08:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3903999999999999e-05 2021-01-15T19:08:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.275254+0000", 2021-01-15T19:08:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071874139999999996 2021-01-15T19:08:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282442+0000", 2021-01-15T19:08:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7597999999999999e-05 2021-01-15T19:08:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282469+0000", 2021-01-15T19:08:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009271049 2021-01-15T19:08:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=107, reqid=client.4418.0:437, at_version=51'43, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:08:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.506410114000001, 2021-01-15T19:08:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003, 2021-01-15T19:08:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274363+0000", 2021-01-15T19:08:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9589999999999998e-05 2021-01-15T19:08:59.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274412+0000", 2021-01-15T19:08:59.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0508000000000001e-05 2021-01-15T19:08:59.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274433+0000", 2021-01-15T19:08:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7353000000000001e-05 2021-01-15T19:08:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274450+0000", 2021-01-15T19:08:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0870000000000001e-06 2021-01-15T19:08:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.274457+0000", 2021-01-15T19:08:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027924999999999998 2021-01-15T19:08:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277250+0000", 2021-01-15T19:08:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3828e-05 2021-01-15T19:08:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.277264+0000", 2021-01-15T19:08:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052077039999999996 2021-01-15T19:08:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282471+0000", 2021-01-15T19:08:59.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3986999999999999e-05 2021-01-15T19:08:59.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282495+0000", 2021-01-15T19:08:59.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081325570000000003 2021-01-15T19:08:59.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.976 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 51/21 ECSubWrite(tid=159, reqid=client.4418.0:448, at_version=51'34, trim_to=0'0, roll_forward_to=50'32))", 2021-01-15T19:08:59.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:59.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.504159242, 2021-01-15T19:08:59.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568, 2021-01-15T19:08:59.977 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.977 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:08:59.977 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:59.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276614+0000", 2021-01-15T19:08:59.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132867 2021-01-15T19:08:59.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276747+0000", 2021-01-15T19:08:59.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6607000000000003e-05 2021-01-15T19:08:59.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276793+0000", 2021-01-15T19:08:59.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9687e-05 2021-01-15T19:08:59.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276813+0000", 2021-01-15T19:08:59.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.374e-06 2021-01-15T19:08:59.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276820+0000", 2021-01-15T19:08:59.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6462999999999998e-05 2021-01-15T19:08:59.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276877+0000", 2021-01-15T19:08:59.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6032000000000002e-05 2021-01-15T19:08:59.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:08:59.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276893+0000", 2021-01-15T19:08:59.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006644418 2021-01-15T19:08:59.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:08:59.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283537+0000", 2021-01-15T19:08:59.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5120000000000003e-05 2021-01-15T19:08:59.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:08:59.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283572+0000", 2021-01-15T19:08:59.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006958568 2021-01-15T19:08:59.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:08:59.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:08:59.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.989 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:08:59.990 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:59.990 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.548563521999998, 2021-01-15T19:08:59.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:08:59.990 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:08:59.991 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:08:59.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:08:59.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:08:59.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:08:59.991 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:08:59.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.992 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:08:59.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:08:59.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:59.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:08:59.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:08:59.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:08:59.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:08:59.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:08:59.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:08:59.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:08:59.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:08:59.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:08:59.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:08:59.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:08:59.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:08:59.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:08:59.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:08:59.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:08:59.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:08:59.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:08:59.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:08:59.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:08:59.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:08:59.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:09:00.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:00.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:09:00.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:09:00.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:00.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:09:00.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:09:00.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:00.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:09:00.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:09:00.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:00.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:09:00.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:09:00.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:00.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:09:00.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:09:00.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.006 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:09:00.007 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:00.007 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.548415504000001, 2021-01-15T19:09:00.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:09:00.007 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:00.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:00.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:00.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:00.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:00.008 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:09:00.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.009 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:00.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:00.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:00.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:00.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:00.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:00.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:09:00.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:00.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:09:00.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:09:00.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:00.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:09:00.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:09:00.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:00.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:09:00.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:09:00.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:00.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:09:00.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:09:00.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:09:00.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:09:00.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:00.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:09:00.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:09:00.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:09:00.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:09:00.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:00.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:09:00.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:09:00.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:09:00.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:09:00.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:00.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:09:00.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:09:00.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:00.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:09:00.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:09:00.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.024 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.025 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.as2 56/21 ECSubWrite(tid=176, reqid=client.4418.0:552, at_version=56'52, trim_to=0'0, roll_forward_to=55'47))", 2021-01-15T19:09:00.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:09:00.025 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.532814384, 2021-01-15T19:09:00.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010275405, 2021-01-15T19:09:00.026 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:00.026 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:00.026 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:00.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:00.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:09:00.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:00.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:00.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.247959+0000", 2021-01-15T19:09:00.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018021319999999999 2021-01-15T19:09:00.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:00.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249761+0000", 2021-01-15T19:09:00.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.951e-05 2021-01-15T19:09:00.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:00.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249790+0000", 2021-01-15T19:09:00.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3495000000000001e-05 2021-01-15T19:09:00.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:00.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249814+0000", 2021-01-15T19:09:00.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7139999999999999e-06 2021-01-15T19:09:00.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:00.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249822+0000", 2021-01-15T19:09:00.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.579e-05 2021-01-15T19:09:00.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249868+0000", 2021-01-15T19:09:00.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0170000000000002e-06 2021-01-15T19:09:00.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:00.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.249877+0000", 2021-01-15T19:09:00.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083158209999999993 2021-01-15T19:09:00.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:00.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258193+0000", 2021-01-15T19:09:00.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0926000000000001e-05 2021-01-15T19:09:00.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:00.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.258234+0000", 2021-01-15T19:09:00.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010275405 2021-01-15T19:09:00.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.037 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.038 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.6s1 61/52 ECSubWrite(tid=210, reqid=client.4418.0:590, at_version=61'36, trim_to=0'0, roll_forward_to=55'32))", 2021-01-15T19:09:00.038 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:49.207749+0000", 2021-01-15T19:09:00.039 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.573024330999999, 2021-01-15T19:09:00.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00653923, 2021-01-15T19:09:00.039 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:00.039 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:00.039 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:00.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:00.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207749+0000", 2021-01-15T19:09:00.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:00.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:00.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207749+0000", 2021-01-15T19:09:00.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163506 2021-01-15T19:09:00.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:00.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207912+0000", 2021-01-15T19:09:00.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1702999999999999e-05 2021-01-15T19:09:00.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:00.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207934+0000", 2021-01-15T19:09:00.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4319e-05 2021-01-15T19:09:00.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:00.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207948+0000", 2021-01-15T19:09:00.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8259999999999993e-06 2021-01-15T19:09:00.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:00.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.207956+0000", 2021-01-15T19:09:00.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9529e-05 2021-01-15T19:09:00.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.208006+0000", 2021-01-15T19:09:00.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9789999999999999e-06 2021-01-15T19:09:00.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:00.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.208014+0000", 2021-01-15T19:09:00.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062396880000000002 2021-01-15T19:09:00.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:00.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.214254+0000", 2021-01-15T19:09:00.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3680000000000003e-05 2021-01-15T19:09:00.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:00.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.214288+0000", 2021-01-15T19:09:00.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00653923 2021-01-15T19:09:00.051 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.051 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.051 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.052 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 65/21 ECSubWrite(tid=160, reqid=client.4418.0:666, at_version=65'75, trim_to=0'0, roll_forward_to=56'73))", 2021-01-15T19:09:00.052 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.266015+0000", 2021-01-15T19:09:00.052 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.5147582320000001, 2021-01-15T19:09:00.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095205849999999998, 2021-01-15T19:09:00.052 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:00.053 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:00.053 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:00.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:00.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.266015+0000", 2021-01-15T19:09:00.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:00.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:00.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.266015+0000", 2021-01-15T19:09:00.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015126079999999999 2021-01-15T19:09:00.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:00.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267527+0000", 2021-01-15T19:09:00.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0474e-05 2021-01-15T19:09:00.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:00.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267558+0000", 2021-01-15T19:09:00.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4299e-05 2021-01-15T19:09:00.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:00.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267572+0000", 2021-01-15T19:09:00.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2490000000000004e-06 2021-01-15T19:09:00.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:00.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267579+0000", 2021-01-15T19:09:00.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4147999999999998e-05 2021-01-15T19:09:00.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267624+0000", 2021-01-15T19:09:00.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8019999999999992e-06 2021-01-15T19:09:00.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:00.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.267631+0000", 2021-01-15T19:09:00.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078664819999999993 2021-01-15T19:09:00.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:00.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.275498+0000", 2021-01-15T19:09:00.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7523e-05 2021-01-15T19:09:00.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:00.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.275535+0000", 2021-01-15T19:09:00.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095205849999999998 2021-01-15T19:09:00.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.064 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.065 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/21 ECSubWrite(tid=197, reqid=client.4418.0:723, at_version=71'96, trim_to=0'0, roll_forward_to=71'94))", 2021-01-15T19:09:00.065 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:59.273849+0000", 2021-01-15T19:09:00.065 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50692446700000005, 2021-01-15T19:09:00.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067329859999999998, 2021-01-15T19:09:00.065 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:00.066 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:00.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:00.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:00.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.273849+0000", 2021-01-15T19:09:00.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:00.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:00.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.273849+0000", 2021-01-15T19:09:00.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080480799999999996 2021-01-15T19:09:00.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:00.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.274653+0000", 2021-01-15T19:09:00.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8677000000000001e-05 2021-01-15T19:09:00.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:00.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.274672+0000", 2021-01-15T19:09:00.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1834000000000001e-05 2021-01-15T19:09:00.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:00.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.274684+0000", 2021-01-15T19:09:00.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8089999999999998e-06 2021-01-15T19:09:00.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:00.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.274690+0000", 2021-01-15T19:09:00.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8966e-05 2021-01-15T19:09:00.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:00.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.274719+0000", 2021-01-15T19:09:00.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9529999999999994e-06 2021-01-15T19:09:00.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:00.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.274729+0000", 2021-01-15T19:09:00.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058164080000000003 2021-01-15T19:09:00.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:00.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.280545+0000", 2021-01-15T19:09:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6531e-05 2021-01-15T19:09:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.280581+0000", 2021-01-15T19:09:00.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067329859999999998 2021-01-15T19:09:00.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.076 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:00.077 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:00.077 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:00.079 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:00.079 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:00.079 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:00.080 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:00.080 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:00.080 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.080 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:00.082 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:00.077+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:00.083 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:00.179 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:00.178+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:00.213 INFO:tasks.rados.rados.0.smithi001.stdout:220: finishing copy_from to smithi00140323-45 2021-01-15T19:09:00.214 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2021-01-15T19:09:00.214 INFO:tasks.rados.rados.0.smithi001.stdout:230: finishing copy_from to smithi00140323-19 2021-01-15T19:09:00.214 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2021-01-15T19:09:00.218 INFO:tasks.rados.rados.0.smithi001.stdout:220: finishing copy_from racing read to smithi00140323-45 2021-01-15T19:09:00.219 INFO:tasks.rados.rados.0.smithi001.stdout:230: finishing copy_from racing read to smithi00140323-19 2021-01-15T19:09:00.226 INFO:tasks.rados.rados.0.smithi001.stdout:234: finishing copy_from to smithi00140323-40 2021-01-15T19:09:00.226 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 40 v 13 (ObjNum 106 snap 17 seq_num 106) dirty exists 2021-01-15T19:09:00.227 INFO:tasks.rados.rados.0.smithi001.stdout:234: finishing copy_from racing read to smithi00140323-40 2021-01-15T19:09:00.265 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:00.266 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:00.275 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:00.286 INFO:tasks.rados.rados.0.smithi001.stdout:220: done (6 left) 2021-01-15T19:09:00.287 INFO:tasks.rados.rados.0.smithi001.stdout:230: done (5 left) 2021-01-15T19:09:00.287 INFO:tasks.rados.rados.0.smithi001.stdout:234: done (4 left) 2021-01-15T19:09:00.288 INFO:tasks.rados.rados.0.smithi001.stdout:265: done (3 left) 2021-01-15T19:09:00.288 INFO:tasks.rados.rados.0.smithi001.stdout:266: done (2 left) 2021-01-15T19:09:00.288 INFO:tasks.rados.rados.0.smithi001.stdout:267: done (1 left) 2021-01-15T19:09:00.289 INFO:tasks.rados.rados.0.smithi001.stdout:268: done (0 left) 2021-01-15T19:09:00.290 INFO:tasks.rados.rados.0.smithi001.stdout:269: rollback oid 5 current snap is 28 2021-01-15T19:09:00.290 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 5 to 13 2021-01-15T19:09:00.290 INFO:tasks.rados.rados.0.smithi001.stdout:270: read oid 35 snap -1 2021-01-15T19:09:00.291 INFO:tasks.rados.rados.0.smithi001.stdout:270: expect (ObjNum 88 snap 16 seq_num 88) 2021-01-15T19:09:00.291 INFO:tasks.rados.rados.0.smithi001.stdout:271: rmattr oid 43 current snap is 28 2021-01-15T19:09:00.292 INFO:tasks.rados.rados.0.smithi001.stdout:271: done (2 left) 2021-01-15T19:09:00.292 INFO:tasks.rados.rados.0.smithi001.stdout:272: snap_create 2021-01-15T19:09:00.293 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:00.284+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:00.295 INFO:tasks.rados.rados.0.smithi001.stdout:270: expect (ObjNum 88 snap 16 seq_num 88) 2021-01-15T19:09:00.336 INFO:tasks.rados.rados.0.smithi001.stdout:269: finishing rollback tid 0 to smithi00140323-5 2021-01-15T19:09:00.336 INFO:tasks.rados.rados.0.smithi001.stdout:269: finishing rollback tid 1 to smithi00140323-5 2021-01-15T19:09:00.336 INFO:tasks.rados.rados.0.smithi001.stdout:269: finishing rollback tid 2 to smithi00140323-5 2021-01-15T19:09:00.337 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 55 (ObjNum 4 snap 0 seq_num 4) dirty exists 2021-01-15T19:09:00.382 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:00.381+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:00.424 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:00.425 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:00.434 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:00.483 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:00.482+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:00.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:09:00.585 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:00.583+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:00.608 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:00.609 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:00.619 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:00.685 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:00.683+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:00.756 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:00.756 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:00.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:00.786 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:00.785+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:00.887 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:00.885+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:00.940 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:00.940 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:00.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:00.991 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:00.990+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:01.092 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:01.091+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:01.124 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:01.125 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:01.132 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:01.218 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:01.216+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:01.289 INFO:tasks.rados.rados.0.smithi001.stdout:269: done (2 left) 2021-01-15T19:09:01.289 INFO:tasks.rados.rados.0.smithi001.stdout:270: done (1 left) 2021-01-15T19:09:01.290 INFO:tasks.rados.rados.0.smithi001.stdout:272: done (0 left) 2021-01-15T19:09:01.290 INFO:tasks.rados.rados.0.smithi001.stdout:273: snap_remove snap 28 2021-01-15T19:09:01.293 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:09:01.294 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:01.292+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:01.310 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:01.310 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:01.320 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:01.405 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:01.403+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:01.492 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:01.492 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:01.499 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:01.498+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:01.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:01.608 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:01.607+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:01.675 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:01.676 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:01.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:01.710 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:01.708+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:01.812 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:01.809+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:01.863 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:01.863 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:01.871 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:01.912 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:01.911+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:02.013 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:02.012+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:02.046 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:02.047 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:02.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:02.082 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:02.147 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:02.146+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:02.194 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:02.194 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:02.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:02.252 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:02.245+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:02.255 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:02.273 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:02.296 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:02.296 INFO:tasks.rados.rados.0.smithi001.stdout:273: done (0 left) 2021-01-15T19:09:02.296 INFO:tasks.rados.rados.0.smithi001.stdout:274: delete oid 17 current snap is 29 2021-01-15T19:09:02.300 INFO:tasks.rados.rados.0.smithi001.stdout:274: done (0 left) 2021-01-15T19:09:02.300 INFO:tasks.rados.rados.0.smithi001.stdout:275: delete oid 30 current snap is 29 2021-01-15T19:09:02.306 INFO:tasks.rados.rados.0.smithi001.stdout:275: done (0 left) 2021-01-15T19:09:02.306 INFO:tasks.rados.rados.0.smithi001.stdout:276: copy_from oid 37 from oid 38 current snap is 29 2021-01-15T19:09:02.306 INFO:tasks.rados.rados.0.smithi001.stdout:277: snap_create 2021-01-15T19:09:02.316 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:02.315+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:02.318 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:02.341 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:02.350 INFO:tasks.rados.rados.0.smithi001.stdout:276: finishing copy_from to smithi00140323-37 2021-01-15T19:09:02.350 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 53 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:02.353 INFO:tasks.rados.rados.0.smithi001.stdout:276: finishing copy_from racing read to smithi00140323-37 2021-01-15T19:09:02.361 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:02.366 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:02.367 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:02.367 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:02.377 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:02.382 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:02.416 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:02.416 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.416 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.416 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.417 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:02.461 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:02.456+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:02.529 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:02.528+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:02.552 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:02.552 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:02.553 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:02.553 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:02.553 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:02.561 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:02.580 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:09:02.580 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:02.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2021-01-15T19:09:02.629 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:02.628+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:02.728 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:02.729 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:02.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:02.729 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:02.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.730 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:185 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [setxattr _header (56) in=7b] snapc 3=[3,2] ondisk+write+known_if_redirected e27)", 2021-01-15T19:09:02.730 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.250240+0000", 2021-01-15T19:09:02.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.476052125999999, 2021-01-15T19:09:02.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012592686000000001, 2021-01-15T19:09:02.732 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.732 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.732 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.732 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.733 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.733 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 185 2021-01-15T19:09:02.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.733 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250240+0000", 2021-01-15T19:09:02.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250240+0000", 2021-01-15T19:09:02.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1121000000000001e-05 2021-01-15T19:09:02.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250291+0000", 2021-01-15T19:09:02.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7689999999999999e-06 2021-01-15T19:09:02.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250298+0000", 2021-01-15T19:09:02.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1284000000000001e-05 2021-01-15T19:09:02.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250309+0000", 2021-01-15T19:09:02.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8729999999999998e-06 2021-01-15T19:09:02.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250314+0000", 2021-01-15T19:09:02.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055389940000000002 2021-01-15T19:09:02.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255853+0000", 2021-01-15T19:09:02.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110605 2021-01-15T19:09:02.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.255964+0000", 2021-01-15T19:09:02.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044564399999999998 2021-01-15T19:09:02.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256409+0000", 2021-01-15T19:09:02.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033779320000000002 2021-01-15T19:09:02.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.259787+0000", 2021-01-15T19:09:02.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029721909999999999 2021-01-15T19:09:02.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262759+0000", 2021-01-15T19:09:02.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3273000000000001e-05 2021-01-15T19:09:02.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.262833+0000", 2021-01-15T19:09:02.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012592686000000001 2021-01-15T19:09:02.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.759 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:186 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e27)", 2021-01-15T19:09:02.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:15.250354+0000", 2021-01-15T19:09:02.761 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.475937690000002, 2021-01-15T19:09:02.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013953569000000001, 2021-01-15T19:09:02.761 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:02.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 186 2021-01-15T19:09:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250354+0000", 2021-01-15T19:09:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250354+0000", 2021-01-15T19:09:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8504000000000003e-05 2021-01-15T19:09:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250393+0000", 2021-01-15T19:09:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3960000000000003e-06 2021-01-15T19:09:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250399+0000", 2021-01-15T19:09:02.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1952e-05 2021-01-15T19:09:02.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250411+0000", 2021-01-15T19:09:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3059999999999999e-06 2021-01-15T19:09:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.250416+0000", 2021-01-15T19:09:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062491630000000003 2021-01-15T19:09:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256665+0000", 2021-01-15T19:09:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101201 2021-01-15T19:09:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:02.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.256766+0000", 2021-01-15T19:09:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006536717 2021-01-15T19:09:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263303+0000", 2021-01-15T19:09:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010614400000000001 2021-01-15T19:09:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.263409+0000", 2021-01-15T19:09:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089918599999999995 2021-01-15T19:09:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:15.264308+0000", 2021-01-15T19:09:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013953569000000001 2021-01-15T19:09:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:364 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [append 0~344064] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191348+0000", 2021-01-15T19:09:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.534944399, 2021-01-15T19:09:02.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013923016999999999, 2021-01-15T19:09:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 364 2021-01-15T19:09:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191348+0000", 2021-01-15T19:09:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191348+0000", 2021-01-15T19:09:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044911899999999999 2021-01-15T19:09:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191797+0000", 2021-01-15T19:09:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.989e-06 2021-01-15T19:09:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191802+0000", 2021-01-15T19:09:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2268e-05 2021-01-15T19:09:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191814+0000", 2021-01-15T19:09:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9809999999999997e-06 2021-01-15T19:09:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191818+0000", 2021-01-15T19:09:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024894549999999998 2021-01-15T19:09:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.194308+0000", 2021-01-15T19:09:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116953 2021-01-15T19:09:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.194425+0000", 2021-01-15T19:09:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00065544400000000004 2021-01-15T19:09:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195080+0000", 2021-01-15T19:09:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005784015 2021-01-15T19:09:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.200864+0000", 2021-01-15T19:09:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043187659999999999 2021-01-15T19:09:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205183+0000", 2021-01-15T19:09:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8027000000000003e-05 2021-01-15T19:09:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205271+0000", 2021-01-15T19:09:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013923016999999999 2021-01-15T19:09:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.792 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:365 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [setxattr _header (57) in=7b] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:09:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.534451036, 2021-01-15T19:09:02.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014071523000000001, 2021-01-15T19:09:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 365 2021-01-15T19:09:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:09:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:09:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5833000000000002e-05 2021-01-15T19:09:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191867+0000", 2021-01-15T19:09:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5580000000000001e-06 2021-01-15T19:09:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191871+0000", 2021-01-15T19:09:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0240000000000003e-06 2021-01-15T19:09:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191877+0000", 2021-01-15T19:09:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4040000000000002e-06 2021-01-15T19:09:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191879+0000", 2021-01-15T19:09:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046935880000000003 2021-01-15T19:09:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196573+0000", 2021-01-15T19:09:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107329 2021-01-15T19:09:02.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196680+0000", 2021-01-15T19:09:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047579799999999999 2021-01-15T19:09:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197156+0000", 2021-01-15T19:09:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003761206 2021-01-15T19:09:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.200917+0000", 2021-01-15T19:09:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044910649999999998 2021-01-15T19:09:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205408+0000", 2021-01-15T19:09:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050471800000000003 2021-01-15T19:09:02.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205913+0000", 2021-01-15T19:09:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014071523000000001 2021-01-15T19:09:02.808 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.809 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:366 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.534393153, 2021-01-15T19:09:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999, 2021-01-15T19:09:02.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.811 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 366 2021-01-15T19:09:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0789999999999999e-05 2021-01-15T19:09:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191920+0000", 2021-01-15T19:09:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2760000000000001e-06 2021-01-15T19:09:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191923+0000", 2021-01-15T19:09:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4840000000000001e-06 2021-01-15T19:09:02.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191930+0000", 2021-01-15T19:09:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2479999999999999e-06 2021-01-15T19:09:02.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191932+0000", 2021-01-15T19:09:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055448800000000003 2021-01-15T19:09:02.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197477+0000", 2021-01-15T19:09:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101512 2021-01-15T19:09:02.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197578+0000", 2021-01-15T19:09:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009253308 2021-01-15T19:09:02.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206832+0000", 2021-01-15T19:09:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011591699999999999 2021-01-15T19:09:02.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206947+0000", 2021-01-15T19:09:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012706729999999999 2021-01-15T19:09:02.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.208218+0000", 2021-01-15T19:09:02.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999 2021-01-15T19:09:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.824 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 50/21 ECSubWriteReply(tid=157, last_complete=50'18, committed=1, applied=1))", 2021-01-15T19:09:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:02.825 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.033363978000001, 2021-01-15T19:09:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001, 2021-01-15T19:09:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4277000000000002e-05 2021-01-15T19:09:02.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692952+0000", 2021-01-15T19:09:02.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015131444000000001 2021-01-15T19:09:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708084+0000", 2021-01-15T19:09:02.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9973999999999999e-05 2021-01-15T19:09:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708104+0000", 2021-01-15T19:09:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4741e-05 2021-01-15T19:09:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708119+0000", 2021-01-15T19:09:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2155000000000004e-05 2021-01-15T19:09:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708181+0000", 2021-01-15T19:09:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7632999999999999e-05 2021-01-15T19:09:02.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708198+0000", 2021-01-15T19:09:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001 2021-01-15T19:09:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.837 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.462161862999999, 2021-01-15T19:09:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:09:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.838 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:09:02.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:02.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:02.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:09:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:09:02.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:09:02.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:09:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:09:02.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:09:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:09:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:09:02.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:09:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:09:02.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:09:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:09:02.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:09:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:09:02.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:09:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:09:02.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:09:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:09:02.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:09:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:09:02.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.853 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.853 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.853 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:02.854 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:02.854 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.461750236, 2021-01-15T19:09:02.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:02.854 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.855 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:02.855 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.855 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.855 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.856 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:02.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.856 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:02.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:02.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:02.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:02.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:02.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:02.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:02.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:02.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:02.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:02.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:02.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:02.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:02.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:02.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:02.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:02.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:02.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:02.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:02.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:02.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:02.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:02.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:02.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:02.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:02.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:02.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:02.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.870 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.871 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:560 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 1f] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:02.871 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.214167+0000", 2021-01-15T19:09:02.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.512125196, 2021-01-15T19:09:02.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012833993, 2021-01-15T19:09:02.872 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.872 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.873 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 560 2021-01-15T19:09:02.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.874 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214167+0000", 2021-01-15T19:09:02.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214167+0000", 2021-01-15T19:09:02.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5794000000000002e-05 2021-01-15T19:09:02.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214213+0000", 2021-01-15T19:09:02.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5909999999999998e-06 2021-01-15T19:09:02.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214220+0000", 2021-01-15T19:09:02.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3059e-05 2021-01-15T19:09:02.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214233+0000", 2021-01-15T19:09:02.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.536e-06 2021-01-15T19:09:02.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214239+0000", 2021-01-15T19:09:02.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047897110000000003 2021-01-15T19:09:02.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.219029+0000", 2021-01-15T19:09:02.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013429499999999999 2021-01-15T19:09:02.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.219163+0000", 2021-01-15T19:09:02.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046991099999999999 2021-01-15T19:09:02.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.219633+0000", 2021-01-15T19:09:02.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003649074 2021-01-15T19:09:02.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.223282+0000", 2021-01-15T19:09:02.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036660640000000001 2021-01-15T19:09:02.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.226948+0000", 2021-01-15T19:09:02.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2957999999999997e-05 2021-01-15T19:09:02.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.227001+0000", 2021-01-15T19:09:02.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012833993 2021-01-15T19:09:02.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.893 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:561 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:02.893 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:44.214286+0000", 2021-01-15T19:09:02.894 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.512006683999999, 2021-01-15T19:09:02.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013434122999999999, 2021-01-15T19:09:02.895 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.895 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.897 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.897 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 561 2021-01-15T19:09:02.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.898 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214286+0000", 2021-01-15T19:09:02.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214286+0000", 2021-01-15T19:09:02.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3198999999999998e-05 2021-01-15T19:09:02.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214329+0000", 2021-01-15T19:09:02.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:09:02.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214336+0000", 2021-01-15T19:09:02.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2753e-05 2021-01-15T19:09:02.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214348+0000", 2021-01-15T19:09:02.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5070000000000002e-06 2021-01-15T19:09:02.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.214353+0000", 2021-01-15T19:09:02.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076004059999999997 2021-01-15T19:09:02.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.221953+0000", 2021-01-15T19:09:02.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103965 2021-01-15T19:09:02.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.222057+0000", 2021-01-15T19:09:02.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053538399999999997 2021-01-15T19:09:02.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.222593+0000", 2021-01-15T19:09:02.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077479700000000003 2021-01-15T19:09:02.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.223367+0000", 2021-01-15T19:09:02.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042704739999999998 2021-01-15T19:09:02.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.227638+0000", 2021-01-15T19:09:02.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1773999999999993e-05 2021-01-15T19:09:02.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:44.227720+0000", 2021-01-15T19:09:02.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013434122999999999 2021-01-15T19:09:02.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.935 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.937 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:589 3.6s0 3:660240bf:::smithi00140323-49:head [append 0~729088] snapc 24=[24,23,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e61)", 2021-01-15T19:09:02.937 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:49.203151+0000", 2021-01-15T19:09:02.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.523141271, 2021-01-15T19:09:02.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012388110000000001, 2021-01-15T19:09:02.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.941 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 589 2021-01-15T19:09:02.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.944 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203151+0000", 2021-01-15T19:09:02.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203151+0000", 2021-01-15T19:09:02.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000747052 2021-01-15T19:09:02.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203898+0000", 2021-01-15T19:09:02.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5559999999999996e-06 2021-01-15T19:09:02.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203908+0000", 2021-01-15T19:09:02.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6201999999999998e-05 2021-01-15T19:09:02.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203924+0000", 2021-01-15T19:09:02.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5369999999999997e-06 2021-01-15T19:09:02.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203932+0000", 2021-01-15T19:09:02.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9701999999999999e-05 2021-01-15T19:09:02.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.203982+0000", 2021-01-15T19:09:02.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0798999999999996e-05 2021-01-15T19:09:02.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.204073+0000", 2021-01-15T19:09:02.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078019299999999999 2021-01-15T19:09:02.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.204853+0000", 2021-01-15T19:09:02.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064241489999999997 2021-01-15T19:09:02.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.211277+0000", 2021-01-15T19:09:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041737470000000002 2021-01-15T19:09:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.215451+0000", 2021-01-15T19:09:02.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8172999999999998e-05 2021-01-15T19:09:02.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:49.215539+0000", 2021-01-15T19:09:02.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012388110000000001 2021-01-15T19:09:02.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.959 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.960 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:634 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~663552] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:02.960 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:02.960 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5262218799999996, 2021-01-15T19:09:02.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952, 2021-01-15T19:09:02.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 634 2021-01-15T19:09:02.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:02.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:02.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081858500000000004 2021-01-15T19:09:02.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200889+0000", 2021-01-15T19:09:02.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2700000000000004e-06 2021-01-15T19:09:02.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200897+0000", 2021-01-15T19:09:02.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7638e-05 2021-01-15T19:09:02.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200915+0000", 2021-01-15T19:09:02.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0730000000000005e-06 2021-01-15T19:09:02.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200923+0000", 2021-01-15T19:09:02.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6312999999999999e-05 2021-01-15T19:09:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200999+0000", 2021-01-15T19:09:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018274900000000001 2021-01-15T19:09:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.201182+0000", 2021-01-15T19:09:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001367224 2021-01-15T19:09:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.202549+0000", 2021-01-15T19:09:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012523378 2021-01-15T19:09:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215073+0000", 2021-01-15T19:09:02.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6226999999999999e-05 2021-01-15T19:09:02.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215169+0000", 2021-01-15T19:09:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131495 2021-01-15T19:09:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215300+0000", 2021-01-15T19:09:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952 2021-01-15T19:09:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.977 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.978 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s2 65/52 ECSubWrite(tid=97, reqid=client.4418.0:627, at_version=65'33, trim_to=0'0, roll_forward_to=55'31))", 2021-01-15T19:09:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.203494+0000", 2021-01-15T19:09:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5227982519999994, 2021-01-15T19:09:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013481198999999999, 2021-01-15T19:09:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.979 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.203494+0000", 2021-01-15T19:09:02.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.203494+0000", 2021-01-15T19:09:02.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000652997 2021-01-15T19:09:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:02.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204147+0000", 2021-01-15T19:09:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1455999999999997e-05 2021-01-15T19:09:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:02.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204188+0000", 2021-01-15T19:09:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5899000000000001e-05 2021-01-15T19:09:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:02.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204214+0000", 2021-01-15T19:09:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8681999999999998e-05 2021-01-15T19:09:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:02.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204233+0000", 2021-01-15T19:09:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0841000000000003e-05 2021-01-15T19:09:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204284+0000", 2021-01-15T19:09:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1681e-05 2021-01-15T19:09:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.204295+0000", 2021-01-15T19:09:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012651252 2021-01-15T19:09:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216947+0000", 2021-01-15T19:09:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8391e-05 2021-01-15T19:09:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216975+0000", 2021-01-15T19:09:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013481198999999999 2021-01-15T19:09:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:02.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:638 3.8s0 3:1009af39:::smithi00140323-25:head [setxattr _header (57) in=7b] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.207660+0000", 2021-01-15T19:09:02.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5186321439999997, 2021-01-15T19:09:02.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013508197, 2021-01-15T19:09:02.996 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:02.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:02.998 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 638 2021-01-15T19:09:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:02.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207660+0000", 2021-01-15T19:09:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207660+0000", 2021-01-15T19:09:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9713000000000003e-05 2021-01-15T19:09:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207710+0000", 2021-01-15T19:09:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5010000000000001e-06 2021-01-15T19:09:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207716+0000", 2021-01-15T19:09:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2828e-05 2021-01-15T19:09:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207729+0000", 2021-01-15T19:09:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1819999999999997e-06 2021-01-15T19:09:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207734+0000", 2021-01-15T19:09:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0090069680000000006 2021-01-15T19:09:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216741+0000", 2021-01-15T19:09:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013609700000000001 2021-01-15T19:09:03.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.216877+0000", 2021-01-15T19:09:03.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052799300000000004 2021-01-15T19:09:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:03.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217405+0000", 2021-01-15T19:09:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026540370000000001 2021-01-15T19:09:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:03.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.220059+0000", 2021-01-15T19:09:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010326790000000001 2021-01-15T19:09:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:03.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221092+0000", 2021-01-15T19:09:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6198999999999993e-05 2021-01-15T19:09:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221168+0000", 2021-01-15T19:09:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013508197 2021-01-15T19:09:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:639 3.8s0 3:1009af39:::smithi00140323-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:03.013 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.5185207510000005, 2021-01-15T19:09:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074, 2021-01-15T19:09:03.014 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:03.015 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 639 2021-01-15T19:09:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:03.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:03.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1959000000000003e-05 2021-01-15T19:09:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207813+0000", 2021-01-15T19:09:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7560000000000002e-06 2021-01-15T19:09:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207820+0000", 2021-01-15T19:09:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207832+0000", 2021-01-15T19:09:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.583e-06 2021-01-15T19:09:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207837+0000", 2021-01-15T19:09:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010014853000000001 2021-01-15T19:09:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217852+0000", 2021-01-15T19:09:03.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0618000000000006e-05 2021-01-15T19:09:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217942+0000", 2021-01-15T19:09:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035136210000000002 2021-01-15T19:09:03.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221456+0000", 2021-01-15T19:09:03.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116369 2021-01-15T19:09:03.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221572+0000", 2021-01-15T19:09:03.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001478222 2021-01-15T19:09:03.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.223051+0000", 2021-01-15T19:09:03.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074 2021-01-15T19:09:03.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.036 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:693 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:03.036 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:03.037 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5451647619999997, 2021-01-15T19:09:03.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668, 2021-01-15T19:09:03.038 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:03.039 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:03.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:03.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:03.040 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:03.041 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 693 2021-01-15T19:09:03.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.042 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:03.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:03.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:03.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:03.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2021-01-15T19:09:03.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181158+0000", 2021-01-15T19:09:03.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5740000000000001e-06 2021-01-15T19:09:03.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181165+0000", 2021-01-15T19:09:03.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2294e-05 2021-01-15T19:09:03.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181177+0000", 2021-01-15T19:09:03.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6979999999999998e-06 2021-01-15T19:09:03.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181185+0000", 2021-01-15T19:09:03.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.101e-05 2021-01-15T19:09:03.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181246+0000", 2021-01-15T19:09:03.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2619999999999999e-05 2021-01-15T19:09:03.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:03.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181258+0000", 2021-01-15T19:09:03.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075312950000000004 2021-01-15T19:09:03.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188790+0000", 2021-01-15T19:09:03.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1936999999999998e-05 2021-01-15T19:09:03.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188872+0000", 2021-01-15T19:09:03.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013017509999999999 2021-01-15T19:09:03.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.201889+0000", 2021-01-15T19:09:03.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668 2021-01-15T19:09:03.056 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.065 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:03.065 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:03.066 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5450777340000004, 2021-01-15T19:09:03.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:03.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:03.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:03.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:03.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:03.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:03.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:03.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:03.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:03.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.545013934, 2021-01-15T19:09:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:03.088 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:03.089 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:711 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~417792] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:09:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:59.238953+0000", 2021-01-15T19:09:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.4873388159999998, 2021-01-15T19:09:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014839906999999999, 2021-01-15T19:09:03.103 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:03.104 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 711 2021-01-15T19:09:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:03.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.238953+0000", 2021-01-15T19:09:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.238953+0000", 2021-01-15T19:09:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061497199999999998 2021-01-15T19:09:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239568+0000", 2021-01-15T19:09:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2340000000000002e-06 2021-01-15T19:09:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239574+0000", 2021-01-15T19:09:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1090000000000001e-05 2021-01-15T19:09:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239585+0000", 2021-01-15T19:09:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7209999999999999e-06 2021-01-15T19:09:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239590+0000", 2021-01-15T19:09:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3004999999999996e-05 2021-01-15T19:09:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239663+0000", 2021-01-15T19:09:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123344 2021-01-15T19:09:03.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239787+0000", 2021-01-15T19:09:03.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099521799999999997 2021-01-15T19:09:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:03.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.240782+0000", 2021-01-15T19:09:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012820557999999999 2021-01-15T19:09:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253603+0000", 2021-01-15T19:09:03.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9769000000000001e-05 2021-01-15T19:09:03.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:03.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253682+0000", 2021-01-15T19:09:03.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011099600000000001 2021-01-15T19:09:03.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253793+0000", 2021-01-15T19:09:03.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014839906999999999 2021-01-15T19:09:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.147 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.148 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:712 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~475136] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:09:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:59.241372+0000", 2021-01-15T19:09:03.151 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.484920298, 2021-01-15T19:09:03.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012725925000000001, 2021-01-15T19:09:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:03.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 712 2021-01-15T19:09:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:03.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.241372+0000", 2021-01-15T19:09:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:03.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.241372+0000", 2021-01-15T19:09:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061849099999999998 2021-01-15T19:09:03.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.241990+0000", 2021-01-15T19:09:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:03.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.241998+0000", 2021-01-15T19:09:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3927999999999999e-05 2021-01-15T19:09:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.242012+0000", 2021-01-15T19:09:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1940000000000003e-06 2021-01-15T19:09:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.242018+0000", 2021-01-15T19:09:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049322599999999999 2021-01-15T19:09:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.246951+0000", 2021-01-15T19:09:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012615599999999999 2021-01-15T19:09:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.247077+0000", 2021-01-15T19:09:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083330299999999997 2021-01-15T19:09:03.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.247910+0000", 2021-01-15T19:09:03.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005941081 2021-01-15T19:09:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253851+0000", 2021-01-15T19:09:03.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014705100000000001 2021-01-15T19:09:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253998+0000", 2021-01-15T19:09:03.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9479000000000001e-05 2021-01-15T19:09:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:03.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.254098+0000", 2021-01-15T19:09:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012725925000000001 2021-01-15T19:09:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:03.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:03.172 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:09:03.174 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 74, 2021-01-15T19:09:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "fe0e1890-043f-40bf-ad11-c5f08349f418", 2021-01-15T19:09:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2021-01-15T19:07:43.378003+0000", 2021-01-15T19:09:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2021-01-15T19:09:02.289721+0000", 2021-01-15T19:09:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2021-01-15T19:07:50.512119+0000", 2021-01-15T19:09:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2021-01-15T19:07:50.512119+0000", 2021-01-15T19:09:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-01-15T19:09:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5804032, 2021-01-15T19:09:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2021-01-15T19:09:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "nodeep-scrub", 2021-01-15T19:09:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2021-01-15T19:09:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2021-01-15T19:09:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2021-01-15T19:09:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2021-01-15T19:09:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2021-01-15T19:09:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2021-01-15T19:09:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-01-15T19:09:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-01-15T19:09:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2021-01-15T19:09:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2021-01-15T19:09:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2021-01-15T19:09:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2021-01-15T19:09:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "luminous", 2021-01-15T19:09:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "pacific", 2021-01-15T19:09:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2021-01-15T19:09:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2021-01-15T19:09:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "device_health_metrics", 2021-01-15T19:09:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:47.545867+0000", 2021-01-15T19:09:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:09:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:09:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:09:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:09:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:09:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:09:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:09:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:09:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:09:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:09:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:09:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:09:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 11, 2021-01-15T19:09:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 11, 2021-01-15T19:09:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 11, 2021-01-15T19:09:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 11, 2021-01-15T19:09:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 11, 2021-01-15T19:09:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:09:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:09:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:09:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:09:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:09:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:09:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:09:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "60", 2021-01-15T19:09:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:09:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:09:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "58", 2021-01-15T19:09:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:09:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:09:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:09:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:09:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:09:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:09:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:09:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:09:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:09:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:09:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:09:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:09:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:09:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:09:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:09:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:09:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:09:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:09:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:09:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:09:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:09:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:09:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:09:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:09:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:09:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:09:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:09:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:09:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:09:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:09:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:09:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:09:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:09:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:09:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2021-01-15T19:09:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2021-01-15T19:09:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:09:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_devicehealth": {} 2021-01-15T19:09:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2021-01-15T19:09:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2021-01-15T19:09:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:54.484638+0000", 2021-01-15T19:09:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:09:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:09:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:09:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:09:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:09:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:09:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:09:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:09:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:09:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:09:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:09:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:09:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 18, 2021-01-15T19:09:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 18, 2021-01-15T19:09:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 18, 2021-01-15T19:09:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 18, 2021-01-15T19:09:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 18, 2021-01-15T19:09:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:09:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:09:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:09:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:09:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:09:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:09:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:09:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "48", 2021-01-15T19:09:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:09:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:09:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "46", 2021-01-15T19:09:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:09:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:09:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:09:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:09:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:09:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:09:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:09:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:09:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:09:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:09:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:09:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:09:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:09:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:09:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:09:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:09:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:09:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:09:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:09:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:09:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:09:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:09:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:09:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:09:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:09:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:09:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:09:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:09:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:09:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:09:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:09:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:09:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:09:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:09:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:09:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:09:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2021-01-15T19:09:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:09:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "unique_pool_0", 2021-01-15T19:09:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:08:08.242953+0000", 2021-01-15T19:09:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2021-01-15T19:09:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2021-01-15T19:09:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 3, 2021-01-15T19:09:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 3, 2021-01-15T19:09:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 2, 2021-01-15T19:09:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 1, 2021-01-15T19:09:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:09:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:09:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:09:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:09:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:09:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:09:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 26, 2021-01-15T19:09:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 26, 2021-01-15T19:09:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 26, 2021-01-15T19:09:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 26, 2021-01-15T19:09:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 26, 2021-01-15T19:09:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:09:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:09:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:09:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:09:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:09:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:09:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:09:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "74", 2021-01-15T19:09:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:09:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:09:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "52", 2021-01-15T19:09:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:09:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:09:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 49, 2021-01-15T19:09:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 74, 2021-01-15T19:09:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:09:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:09:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:09:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:09:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:09:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:09:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:09:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:09:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:09:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:09:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:09:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:09:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:09:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:09:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:09:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:09:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "isaprofile", 2021-01-15T19:09:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:09:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:09:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:09:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:09:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:09:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:09:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:09:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:09:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:09:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:09:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 8192, 2021-01-15T19:09:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:09:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:09:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:09:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:09:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "rados": {} 2021-01-15T19:09:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:09:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:09:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e5b26f89-5c99-4764-9183-f3bb27c118f5", 2021-01-15T19:09:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 63, 2021-01-15T19:09:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2021-01-15T19:09:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2021-01-15T19:09:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2021-01-15T19:09:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2021-01-15T19:09:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2021-01-15T19:09:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2021-01-15T19:09:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2021-01-15T19:09:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2021-01-15T19:09:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:09:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/34833", 2021-01-15T19:09:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/34833", 2021-01-15T19:09:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/34833", 2021-01-15T19:09:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/34833", 2021-01-15T19:09:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:09:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "99c0d455-1bd6-43aa-bfec-b6bddf87a35e", 2021-01-15T19:09:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.258 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 63, 2021-01-15T19:09:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2021-01-15T19:09:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2021-01-15T19:09:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2021-01-15T19:09:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2021-01-15T19:09:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2021-01-15T19:09:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2021-01-15T19:09:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2021-01-15T19:09:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2021-01-15T19:09:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:09:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/34834", 2021-01-15T19:09:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/34834", 2021-01-15T19:09:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/34834", 2021-01-15T19:09:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/34834", 2021-01-15T19:09:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:09:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "71fe6c4b-3d73-4e07-b49d-98596ca7d9da", 2021-01-15T19:09:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 63, 2021-01-15T19:09:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2021-01-15T19:09:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2021-01-15T19:09:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2021-01-15T19:09:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2021-01-15T19:09:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2021-01-15T19:09:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2021-01-15T19:09:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2021-01-15T19:09:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2021-01-15T19:09:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:09:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6819/34863", 2021-01-15T19:09:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6823/34863", 2021-01-15T19:09:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/34863", 2021-01-15T19:09:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/34863", 2021-01-15T19:09:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:09:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8a6b018a-3a48-4a01-8526-a386f62ecb36", 2021-01-15T19:09:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.298 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 63, 2021-01-15T19:09:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2021-01-15T19:09:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2021-01-15T19:09:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2021-01-15T19:09:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2021-01-15T19:09:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2021-01-15T19:09:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2021-01-15T19:09:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2021-01-15T19:09:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2021-01-15T19:09:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:09:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6818/34864", 2021-01-15T19:09:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6822/34864", 2021-01-15T19:09:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/34864", 2021-01-15T19:09:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/34864", 2021-01-15T19:09:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:09:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "ff089622-e642-4bd0-901c-60efdb87a549", 2021-01-15T19:09:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 63, 2021-01-15T19:09:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6800", 2021-01-15T19:09:03.329 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.331 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6801", 2021-01-15T19:09:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6802", 2021-01-15T19:09:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6803", 2021-01-15T19:09:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6806", 2021-01-15T19:09:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6807", 2021-01-15T19:09:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6804", 2021-01-15T19:09:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6805", 2021-01-15T19:09:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:09:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6801/34829", 2021-01-15T19:09:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6803/34829", 2021-01-15T19:09:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6807/34829", 2021-01-15T19:09:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6805/34829", 2021-01-15T19:09:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.350 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:09:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "3c57c228-dfaa-4e33-bded-27da0048d141", 2021-01-15T19:09:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 57, 2021-01-15T19:09:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6809", 2021-01-15T19:09:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6811", 2021-01-15T19:09:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6813", 2021-01-15T19:09:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6815", 2021-01-15T19:09:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6821", 2021-01-15T19:09:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6823", 2021-01-15T19:09:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.426 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.427 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.427 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.428 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6817", 2021-01-15T19:09:03.429 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.430 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6819", 2021-01-15T19:09:03.437 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:09:03.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.454 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.455 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6811/34830", 2021-01-15T19:09:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6815/34830", 2021-01-15T19:09:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6823/34830", 2021-01-15T19:09:03.456 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6819/34830", 2021-01-15T19:09:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.460 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:09:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "201f889f-dac1-4a26-a285-7a0ec21c3646", 2021-01-15T19:09:03.461 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.462 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.463 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.468 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.469 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.470 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 57, 2021-01-15T19:09:03.477 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.479 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.480 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.487 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.489 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6808", 2021-01-15T19:09:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6810", 2021-01-15T19:09:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6812", 2021-01-15T19:09:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6814", 2021-01-15T19:09:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6820", 2021-01-15T19:09:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.514 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6822", 2021-01-15T19:09:03.518 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.519 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.520 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.522 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6816", 2021-01-15T19:09:03.524 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.527 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6818", 2021-01-15T19:09:03.532 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34831 2021-01-15T19:09:03.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.535 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.538 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6810/34831", 2021-01-15T19:09:03.539 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6814/34831", 2021-01-15T19:09:03.541 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6822/34831", 2021-01-15T19:09:03.543 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6818/34831", 2021-01-15T19:09:03.543 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.545 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.550 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.552 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.558 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:09:03.559 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "79ffd413-a9d3-4180-9691-d834d89c3f2e", 2021-01-15T19:09:03.560 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:09:03.561 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:09:03.561 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:09:03.565 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:09:03.565 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:09:03.566 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:09:03.566 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:09:03.566 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 63, 2021-01-15T19:09:03.566 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:09:03.567 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:09:03.567 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:09:03.567 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.568 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6824", 2021-01-15T19:09:03.568 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.569 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6825", 2021-01-15T19:09:03.569 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.570 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.570 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:09:03.570 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.571 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6826", 2021-01-15T19:09:03.571 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.574 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6827", 2021-01-15T19:09:03.574 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.574 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.575 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:09:03.575 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.576 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6830", 2021-01-15T19:09:03.576 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.577 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6831", 2021-01-15T19:09:03.577 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.578 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:09:03.578 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:09:03.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:09:03.579 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6828", 2021-01-15T19:09:03.579 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:09:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6829", 2021-01-15T19:09:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:09:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.580 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6825/34874", 2021-01-15T19:09:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6827/34874", 2021-01-15T19:09:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6831/34874", 2021-01-15T19:09:03.581 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6829/34874", 2021-01-15T19:09:03.582 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:09:03.582 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:09:03.582 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:09:03.582 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.583 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.583 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2021-01-15T19:09:03.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.583 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:09:03.583 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.584 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.584 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.584 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.584 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.584 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.060279+0000", 2021-01-15T19:09:03.585 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.585 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:09:03.586 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.587 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.587 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.587 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.587 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.588 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.065132+0000", 2021-01-15T19:09:03.588 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:09:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.589 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.853664+0000", 2021-01-15T19:09:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:09:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.248039+0000", 2021-01-15T19:09:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.593 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:09:03.593 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.593 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.593 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.594 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.594 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.594 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.334498+0000", 2021-01-15T19:09:03.594 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.595 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:09:03.595 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.595 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.595 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.596 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.596 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.596 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.484106+0000", 2021-01-15T19:09:03.596 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.597 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:09:03.597 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.597 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.597 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.598 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.598 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.599 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.837613+0000", 2021-01-15T19:09:03.599 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.599 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:09:03.600 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:09:03.600 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:09:03.600 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:09:03.600 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:09:03.600 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:09:03.601 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:08:49.696879+0000", 2021-01-15T19:09:03.601 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:09:03.601 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.601 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.601 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2021-01-15T19:09:03.602 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [ 2021-01-15T19:09:03.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.602 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.1", 2021-01-15T19:09:03.602 INFO:teuthology.orchestra.run.smithi001.stdout: "mappings": [ 2021-01-15T19:09:03.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.603 INFO:teuthology.orchestra.run.smithi001.stdout: "from": 5, 2021-01-15T19:09:03.603 INFO:teuthology.orchestra.run.smithi001.stdout: "to": 2 2021-01-15T19:09:03.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.603 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.604 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.604 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2021-01-15T19:09:03.604 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2021-01-15T19:09:03.604 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2021-01-15T19:09:03.604 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2021-01-15T19:09:03.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2021-01-15T19:09:03.605 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:09:03.605 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:09:03.605 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:09:03.605 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2021-01-15T19:09:03.606 INFO:teuthology.orchestra.run.smithi001.stdout: "ruleset-failure-domain": "osd", 2021-01-15T19:09:03.606 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:09:03.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.606 INFO:teuthology.orchestra.run.smithi001.stdout: "isaprofile": { 2021-01-15T19:09:03.606 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-device-class": "", 2021-01-15T19:09:03.607 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:09:03.607 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-root": "default", 2021-01-15T19:09:03.607 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:09:03.607 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:09:03.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "isaprofile", 2021-01-15T19:09:03.608 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "isa", 2021-01-15T19:09:03.608 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:09:03.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.609 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [ 2021-01-15T19:09:03.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.609 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:09:03.609 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2021-01-15T19:09:03.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.610 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 16, 2021-01-15T19:09:03.610 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:09:03.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.611 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 20, 2021-01-15T19:09:03.611 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:09:03.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.611 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 28, 2021-01-15T19:09:03.613 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:09:03.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.615 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 32, 2021-01-15T19:09:03.615 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:09:03.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.617 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 35, 2021-01-15T19:09:03.618 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:09:03.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:03.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.619 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 48, 2021-01-15T19:09:03.620 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:09:03.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.620 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.621 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.622 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [ 2021-01-15T19:09:03.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.623 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:09:03.624 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2021-01-15T19:09:03.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:03.625 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 48, 2021-01-15T19:09:03.625 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:09:03.626 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.627 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:03.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.627 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:03.628 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2021-01-15T19:09:03.628 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2021-01-15T19:09:03.629 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2021-01-15T19:09:03.629 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2021-01-15T19:09:03.629 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2021-01-15T19:09:03.630 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2021-01-15T19:09:03.631 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2021-01-15T19:09:03.631 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2021-01-15T19:09:03.631 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2021-01-15T19:09:03.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:03.632 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:03.634 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:03.638 INFO:tasks.thrashosds.thrasher:j is {'epoch': 74, 'fsid': 'fe0e1890-043f-40bf-ad11-c5f08349f418', 'created': '2021-01-15T19:07:43.378003+0000', 'modified': '2021-01-15T19:09:02.289721+0000', 'last_up_change': '2021-01-15T19:07:50.512119+0000', 'last_in_change': '2021-01-15T19:07:50.512119+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '58', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 18, 'pg_placement_num': 18, 'pg_placement_num_target': 18, 'pg_num_target': 18, 'pg_num_pending': 18, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '46', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '52', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 49, 'snap_epoch': 74, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e5b26f89-5c99-4764-9183-f3bb27c118f5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 34833}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 34833}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 34833}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 34833}]}, 'public_addr': '172.21.15.1:6801/34833', 'cluster_addr': '172.21.15.1:6803/34833', 'heartbeat_back_addr': '172.21.15.1:6807/34833', 'heartbeat_front_addr': '172.21.15.1:6805/34833', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '99c0d455-1bd6-43aa-bfec-b6bddf87a35e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 34834}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 34834}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 34834}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 34834}]}, 'public_addr': '172.21.15.1:6809/34834', 'cluster_addr': '172.21.15.1:6811/34834', 'heartbeat_back_addr': '172.21.15.1:6815/34834', 'heartbeat_front_addr': '172.21.15.1:6813/34834', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '71fe6c4b-3d73-4e07-b49d-98596ca7d9da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6817', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 34863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6821', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 34863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 34863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 34863}]}, 'public_addr': '172.21.15.1:6819/34863', 'cluster_addr': '172.21.15.1:6823/34863', 'heartbeat_back_addr': '172.21.15.1:6831/34863', 'heartbeat_front_addr': '172.21.15.1:6828/34863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8a6b018a-3a48-4a01-8526-a386f62ecb36', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6818', 'nonce': 34864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6822', 'nonce': 34864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 34864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 34864}]}, 'public_addr': '172.21.15.1:6818/34864', 'cluster_addr': '172.21.15.1:6822/34864', 'heartbeat_back_addr': '172.21.15.1:6829/34864', 'heartbeat_front_addr': '172.21.15.1:6825/34864', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ff089622-e642-4bd0-901c-60efdb87a549', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 34829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 34829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 34829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 34829}]}, 'public_addr': '172.21.15.136:6801/34829', 'cluster_addr': '172.21.15.136:6803/34829', 'heartbeat_back_addr': '172.21.15.136:6807/34829', 'heartbeat_front_addr': '172.21.15.136:6805/34829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3c57c228-dfaa-4e33-bded-27da0048d141', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 34830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 34830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6821', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 34830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6817', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 34830}]}, 'public_addr': '172.21.15.136:6811/34830', 'cluster_addr': '172.21.15.136:6815/34830', 'heartbeat_back_addr': '172.21.15.136:6823/34830', 'heartbeat_front_addr': '172.21.15.136:6819/34830', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '201f889f-dac1-4a26-a285-7a0ec21c3646', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6810', 'nonce': 34831}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6814', 'nonce': 34831}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6822', 'nonce': 34831}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 34831}, {'type': 'v1', 'addr': '172.21.15.136:6818', 'nonce': 34831}]}, 'public_addr': '172.21.15.136:6810/34831', 'cluster_addr': '172.21.15.136:6814/34831', 'heartbeat_back_addr': '172.21.15.136:6822/34831', 'heartbeat_front_addr': '172.21.15.136:6818/34831', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '79ffd413-a9d3-4180-9691-d834d89c3f2e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 34874}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 34874}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 34874}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 34874}]}, 'public_addr': '172.21.15.136:6825/34874', 'cluster_addr': '172.21.15.136:6827/34874', 'heartbeat_back_addr': '172.21.15.136:6831/34874', 'heartbeat_front_addr': '172.21.15.136:6829/34874', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.060279+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.065132+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.853664+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.248039+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.334498+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.484106+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.837613+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:08:49.696879+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.1', 'mappings': [{'from': 5, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'isaprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '2', 'm': '1', 'name': 'isaprofile', 'plugin': 'isa', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 1}, {'begin': 20, 'length': 1}, {'begin': 28, 'length': 1}, {'begin': 32, 'length': 2}, {'begin': 35, 'length': 2}, {'begin': 48, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 48, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2021-01-15T19:09:03.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:09:03.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:09:03.647 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:03.175+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:03.665 INFO:tasks.rados.rados.0.smithi001.stdout:276: done (1 left) 2021-01-15T19:09:03.666 INFO:tasks.rados.rados.0.smithi001.stdout:277: done (0 left) 2021-01-15T19:09:03.667 INFO:tasks.rados.rados.0.smithi001.stdout:278: delete oid 46 current snap is 30 2021-01-15T19:09:03.667 INFO:tasks.rados.rados.0.smithi001.stdout:278: done (0 left) 2021-01-15T19:09:03.667 INFO:tasks.rados.rados.0.smithi001.stdout:279: read oid 7 snap -1 2021-01-15T19:09:03.668 INFO:tasks.rados.rados.0.smithi001.stdout:279: expect (ObjNum 121 snap 27 seq_num 121) 2021-01-15T19:09:03.670 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 46 current snap is 30 2021-01-15T19:09:03.671 INFO:tasks.rados.rados.0.smithi001.stdout:280: seq_num 126 ranges {0=516096,516096=630784,1146880=614400,1761280=360448} 2021-01-15T19:09:03.671 INFO:tasks.rados.rados.0.smithi001.stdout:280: writing smithi00140323-46 from 0 to 516096 tid 1 2021-01-15T19:09:03.672 INFO:tasks.rados.rados.0.smithi001.stdout:280: writing smithi00140323-46 from 516096 to 1146880 tid 2 2021-01-15T19:09:03.673 INFO:tasks.rados.rados.0.smithi001.stdout:280: writing smithi00140323-46 from 1146880 to 1761280 tid 3 2021-01-15T19:09:03.673 INFO:tasks.rados.rados.0.smithi001.stdout:280: writing smithi00140323-46 from 1761280 to 2121728 tid 4 2021-01-15T19:09:03.673 INFO:tasks.rados.rados.0.smithi001.stdout:281: rmattr oid 33 current snap is 30 2021-01-15T19:09:03.674 INFO:tasks.rados.rados.0.smithi001.stdout:280: finishing write tid 1 to smithi00140323-46 2021-01-15T19:09:03.674 INFO:tasks.rados.rados.0.smithi001.stdout:279: expect (ObjNum 121 snap 27 seq_num 121) 2021-01-15T19:09:03.674 INFO:tasks.rados.rados.0.smithi001.stdout:280: finishing write tid 2 to smithi00140323-46 2021-01-15T19:09:03.675 INFO:tasks.rados.rados.0.smithi001.stdout:280: finishing write tid 3 to smithi00140323-46 2021-01-15T19:09:03.676 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 100 (ObjNum 102 snap 16 seq_num 102) dirty exists 2021-01-15T19:09:03.676 INFO:tasks.rados.rados.0.smithi001.stdout:280: finishing write tid 4 to smithi00140323-46 2021-01-15T19:09:03.676 INFO:tasks.rados.rados.0.smithi001.stdout:280: finishing write tid 5 to smithi00140323-46 2021-01-15T19:09:03.677 INFO:tasks.rados.rados.0.smithi001.stdout:279: done (2 left) 2021-01-15T19:09:03.677 INFO:tasks.rados.rados.0.smithi001.stdout:281: done (1 left) 2021-01-15T19:09:03.677 INFO:tasks.rados.rados.0.smithi001.stdout:282: read oid 39 snap 14 2021-01-15T19:09:03.678 INFO:tasks.rados.rados.0.smithi001.stdout:282: expect (ObjNum 76 snap 12 seq_num 76) 2021-01-15T19:09:03.678 INFO:tasks.rados.rados.0.smithi001.stdout:283: snap_remove snap 26 2021-01-15T19:09:03.679 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:03.642+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:03.746 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:03.742+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:03.810 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:03.811 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:03.811 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:03.811 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:03.827 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:03.847 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:03.844+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:03.955 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:03.950+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:03.955 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:03.957 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:03.968 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:04.009 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:09:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:09:04.019 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2021-01-15T19:09:03.518354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":122056704,"num_objects":108,"num_object_clones":61,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":3,"num_read":253,"num_read_kb":91166,"num_write":372,"num_write_kb":102627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":12673024,"num_keys_recovered":1,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1184,"ondisk_log_size":1184,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269076,"kb_used_data":191188,"kb_used_omap":0,"kb_used_meta":77824,"kb_avail":754705644,"statfs":{"total":773094113280,"available":772818579456,"internally_reserved":0,"allocated":195776512,"data_stored":258718252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":79691776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13500416,"num_objects":-1,"num_object_clones":0,"num_object_copies":-3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":-2,"num_read":21,"num_read_kb":19332,"num_write":30,"num_write_kb":14944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000921"},"pg_stats":[{"pgid":"3.19","version":"71'40","reported_seq":"89","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.695789+0000","last_change":"2021-01-15T19:08:59.695789+0000","last_active":"2021-01-15T19:08:59.695789+0000","last_peered":"2021-01-15T19:08:59.695789+0000","last_clean":"2021-01-15T19:08:59.695789+0000","last_became_active":"2021-01-15T19:08:46.196966+0000","last_became_peered":"2021-01-15T19:08:46.196966+0000","last_unstale":"2021-01-15T19:08:59.695789+0000","last_undegraded":"2021-01-15T19:08:59.695789+0000","last_fullsized":"2021-01-15T19:08:59.695789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6737920,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":11,"num_read_kb":1468,"num_write":11,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2678784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.18","version":"51'34","reported_seq":"75","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.188780+0000","last_change":"2021-01-15T19:08:59.189211+0000","last_active":"2021-01-15T19:08:59.188780+0000","last_peered":"2021-01-15T19:08:59.188780+0000","last_clean":"2021-01-15T19:08:59.188780+0000","last_became_active":"2021-01-15T19:08:46.190665+0000","last_became_peered":"2021-01-15T19:08:46.190665+0000","last_unstale":"2021-01-15T19:08:59.188780+0000","last_undegraded":"2021-01-15T19:08:59.188780+0000","last_fullsized":"2021-01-15T19:08:59.188780+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"51'34","last_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_deep_scrub":"51'34","last_deep_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.694318+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":761856,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.17","version":"50'30","reported_seq":"56","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.188872+0000","last_change":"2021-01-15T19:08:59.189366+0000","last_active":"2021-01-15T19:08:59.188872+0000","last_peered":"2021-01-15T19:08:59.188872+0000","last_clean":"2021-01-15T19:08:59.188872+0000","last_became_active":"2021-01-15T19:08:46.192768+0000","last_became_peered":"2021-01-15T19:08:46.192768+0000","last_unstale":"2021-01-15T19:08:59.188872+0000","last_undegraded":"2021-01-15T19:08:59.188872+0000","last_fullsized":"2021-01-15T19:08:59.188872+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1519616,"num_objects":1,"num_object_clones":1,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":8,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.16","version":"58'32","reported_seq":"89","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.195904+0000","last_change":"2021-01-15T19:08:59.195904+0000","last_active":"2021-01-15T19:08:59.195904+0000","last_peered":"2021-01-15T19:08:59.195904+0000","last_clean":"2021-01-15T19:08:59.195904+0000","last_became_active":"2021-01-15T19:08:46.188311+0000","last_became_peered":"2021-01-15T19:08:46.188311+0000","last_unstale":"2021-01-15T19:08:59.195904+0000","last_undegraded":"2021-01-15T19:08:59.195904+0000","last_fullsized":"2021-01-15T19:08:59.195904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'32","last_scrub_stamp":"2021-01-15T19:08:51.873289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.873289+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":139264,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.15","version":"71'32","reported_seq":"82","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.705090+0000","last_change":"2021-01-15T19:08:59.705090+0000","last_active":"2021-01-15T19:08:59.705090+0000","last_peered":"2021-01-15T19:08:59.705090+0000","last_clean":"2021-01-15T19:08:59.705090+0000","last_became_active":"2021-01-15T19:08:46.190760+0000","last_became_peered":"2021-01-15T19:08:46.190760+0000","last_unstale":"2021-01-15T19:08:59.705090+0000","last_undegraded":"2021-01-15T19:08:59.705090+0000","last_fullsized":"2021-01-15T19:08:59.705090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2748416,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":20,"num_write_kb":2684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":696320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.14","version":"71'41","reported_seq":"86","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:09:00.227502+0000","last_change":"2021-01-15T19:09:00.194492+0000","last_active":"2021-01-15T19:09:00.227502+0000","last_peered":"2021-01-15T19:09:00.227502+0000","last_clean":"2021-01-15T19:09:00.227502+0000","last_became_active":"2021-01-15T19:08:46.193553+0000","last_became_peered":"2021-01-15T19:08:46.193553+0000","last_unstale":"2021-01-15T19:09:00.227502+0000","last_undegraded":"2021-01-15T19:09:00.227502+0000","last_fullsized":"2021-01-15T19:09:00.227502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4898816,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":4024,"num_write":11,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2801664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.13","version":"0'0","reported_seq":"76","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.190457+0000","last_change":"2021-01-15T19:08:59.190457+0000","last_active":"2021-01-15T19:08:59.190457+0000","last_peered":"2021-01-15T19:08:59.190457+0000","last_clean":"2021-01-15T19:08:59.190457+0000","last_became_active":"2021-01-15T19:08:43.203849+0000","last_became_peered":"2021-01-15T19:08:43.203849+0000","last_unstale":"2021-01-15T19:08:59.190457+0000","last_undegraded":"2021-01-15T19:08:59.190457+0000","last_fullsized":"2021-01-15T19:08:59.190457+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"3.12","version":"69'38","reported_seq":"105","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.186020+0000","last_change":"2021-01-15T19:08:59.186497+0000","last_active":"2021-01-15T19:08:59.186020+0000","last_peered":"2021-01-15T19:08:59.186020+0000","last_clean":"2021-01-15T19:08:59.186020+0000","last_became_active":"2021-01-15T19:08:46.189687+0000","last_became_peered":"2021-01-15T19:08:46.189687+0000","last_unstale":"2021-01-15T19:08:59.186020+0000","last_undegraded":"2021-01-15T19:08:59.186020+0000","last_fullsized":"2021-01-15T19:08:59.186020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4321280,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":4829,"num_write":16,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":3809280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"1.4","version":"14'2","reported_seq":"26","reported_epoch":"72","state":"active+clean","last_fresh":"2021-01-15T19:09:00.285551+0000","last_change":"2021-01-15T19:08:53.068011+0000","last_active":"2021-01-15T19:09:00.285551+0000","last_peered":"2021-01-15T19:09:00.285551+0000","last_clean":"2021-01-15T19:09:00.285551+0000","last_became_active":"2021-01-15T19:08:52.197215+0000","last_became_peered":"2021-01-15T19:08:52.197215+0000","last_unstale":"2021-01-15T19:09:00.285551+0000","last_undegraded":"2021-01-15T19:09:00.285551+0000","last_fullsized":"2021-01-15T19:09:00.285551+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"71'54","reported_seq":"203","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:09:00.199355+0000","last_change":"2021-01-15T19:09:00.199355+0000","last_active":"2021-01-15T19:09:00.199355+0000","last_peered":"2021-01-15T19:09:00.199355+0000","last_clean":"2021-01-15T19:09:00.199355+0000","last_became_active":"2021-01-15T19:08:41.227163+0000","last_became_peered":"2021-01-15T19:08:41.227163+0000","last_unstale":"2021-01-15T19:09:00.199355+0000","last_undegraded":"2021-01-15T19:09:00.199355+0000","last_fullsized":"2021-01-15T19:09:00.199355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3837952,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":208,"num_write":16,"num_write_kb":4080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":"68","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.189719+0000","last_change":"2021-01-15T19:08:35.209452+0000","last_active":"2021-01-15T19:08:59.189719+0000","last_peered":"2021-01-15T19:08:59.189719+0000","last_clean":"2021-01-15T19:08:59.189719+0000","last_became_active":"2021-01-15T19:08:35.208618+0000","last_became_peered":"2021-01-15T19:08:35.208618+0000","last_unstale":"2021-01-15T19:08:59.189719+0000","last_undegraded":"2021-01-15T19:08:59.189719+0000","last_fullsized":"2021-01-15T19:08:59.189719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"64'3","reported_seq":"38","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.193207+0000","last_change":"2021-01-15T19:08:52.906711+0000","last_active":"2021-01-15T19:08:59.193207+0000","last_peered":"2021-01-15T19:08:59.193207+0000","last_clean":"2021-01-15T19:08:59.193207+0000","last_became_active":"2021-01-15T19:08:52.201512+0000","last_became_peered":"2021-01-15T19:08:52.201512+0000","last_unstale":"2021-01-15T19:08:59.193207+0000","last_undegraded":"2021-01-15T19:08:59.193207+0000","last_fullsized":"2021-01-15T19:08:59.193207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.906630+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"75","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.191384+0000","last_change":"2021-01-15T19:08:38.030592+0000","last_active":"2021-01-15T19:08:59.191384+0000","last_peered":"2021-01-15T19:08:59.191384+0000","last_clean":"2021-01-15T19:08:59.191384+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:08:59.191384+0000","last_undegraded":"2021-01-15T19:08:59.191384+0000","last_fullsized":"2021-01-15T19:08:59.191384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.030523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.030523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"69'39","reported_seq":"192","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.696895+0000","last_change":"2021-01-15T19:08:59.696895+0000","last_active":"2021-01-15T19:08:59.696895+0000","last_peered":"2021-01-15T19:08:59.696895+0000","last_clean":"2021-01-15T19:08:59.696895+0000","last_became_active":"2021-01-15T19:08:41.236424+0000","last_became_peered":"2021-01-15T19:08:41.236424+0000","last_unstale":"2021-01-15T19:08:59.696895+0000","last_undegraded":"2021-01-15T19:08:59.696895+0000","last_fullsized":"2021-01-15T19:08:59.696895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'37","last_scrub_stamp":"2021-01-15T19:08:54.947352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:54.947352+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4710400,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":13801,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"1.6","version":"14'2","reported_seq":"27","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.190673+0000","last_change":"2021-01-15T19:08:52.210882+0000","last_active":"2021-01-15T19:08:59.190673+0000","last_peered":"2021-01-15T19:08:59.190673+0000","last_clean":"2021-01-15T19:08:59.190673+0000","last_became_active":"2021-01-15T19:08:52.210341+0000","last_became_peered":"2021-01-15T19:08:52.210341+0000","last_unstale":"2021-01-15T19:08:59.190673+0000","last_undegraded":"2021-01-15T19:08:59.190673+0000","last_fullsized":"2021-01-15T19:08:59.190673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"71'47","reported_seq":"202","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.690804+0000","last_change":"2021-01-15T19:08:59.691173+0000","last_active":"2021-01-15T19:08:59.690804+0000","last_peered":"2021-01-15T19:08:59.690804+0000","last_clean":"2021-01-15T19:08:59.690804+0000","last_became_active":"2021-01-15T19:08:41.239519+0000","last_became_peered":"2021-01-15T19:08:41.239519+0000","last_unstale":"2021-01-15T19:08:59.690804+0000","last_undegraded":"2021-01-15T19:08:59.690804+0000","last_fullsized":"2021-01-15T19:08:59.690804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'35","last_scrub_stamp":"2021-01-15T19:08:51.059372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.059372+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2686976,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":"72","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.190249+0000","last_change":"2021-01-15T19:08:55.065884+0000","last_active":"2021-01-15T19:08:59.190249+0000","last_peered":"2021-01-15T19:08:59.190249+0000","last_clean":"2021-01-15T19:08:59.190249+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:08:59.190249+0000","last_undegraded":"2021-01-15T19:08:59.190249+0000","last_fullsized":"2021-01-15T19:08:59.190249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_clean_scrub_stamp":"2021-01-15T19:08:55.065810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"85","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.190828+0000","last_change":"2021-01-15T19:08:52.202853+0000","last_active":"2021-01-15T19:08:59.190828+0000","last_peered":"2021-01-15T19:08:59.190828+0000","last_clean":"2021-01-15T19:08:59.190828+0000","last_became_active":"2021-01-15T19:08:52.202463+0000","last_became_peered":"2021-01-15T19:08:52.202463+0000","last_unstale":"2021-01-15T19:08:59.190828+0000","last_undegraded":"2021-01-15T19:08:59.190828+0000","last_fullsized":"2021-01-15T19:08:59.190828+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"71'66","reported_seq":"216","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.693578+0000","last_change":"2021-01-15T19:08:59.694038+0000","last_active":"2021-01-15T19:08:59.693578+0000","last_peered":"2021-01-15T19:08:59.693578+0000","last_clean":"2021-01-15T19:08:59.693578+0000","last_became_active":"2021-01-15T19:08:41.240170+0000","last_became_peered":"2021-01-15T19:08:41.240170+0000","last_unstale":"2021-01-15T19:08:59.693578+0000","last_undegraded":"2021-01-15T19:08:59.693578+0000","last_fullsized":"2021-01-15T19:08:59.693578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'47","last_scrub_stamp":"2021-01-15T19:08:53.015941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.015941+0000","log_size":66,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6692864,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":3816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":"72","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.189239+0000","last_change":"2021-01-15T19:08:43.003842+0000","last_active":"2021-01-15T19:08:59.189239+0000","last_peered":"2021-01-15T19:08:59.189239+0000","last_clean":"2021-01-15T19:08:59.189239+0000","last_became_active":"2021-01-15T19:08:35.210980+0000","last_became_peered":"2021-01-15T19:08:35.210980+0000","last_unstale":"2021-01-15T19:08:59.189239+0000","last_undegraded":"2021-01-15T19:08:59.189239+0000","last_fullsized":"2021-01-15T19:08:59.189239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:43.003765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:43.003765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"168","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.696324+0000","last_change":"2021-01-15T19:08:59.696324+0000","last_active":"2021-01-15T19:08:59.696324+0000","last_peered":"2021-01-15T19:08:59.696324+0000","last_clean":"2021-01-15T19:08:59.696324+0000","last_became_active":"2021-01-15T19:08:41.234856+0000","last_became_peered":"2021-01-15T19:08:41.234856+0000","last_unstale":"2021-01-15T19:08:59.696324+0000","last_undegraded":"2021-01-15T19:08:59.696324+0000","last_fullsized":"2021-01-15T19:08:59.696324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":987136,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1610,"num_write":5,"num_write_kb":964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"1.0","version":"14'2","reported_seq":"81","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.185125+0000","last_change":"2021-01-15T19:08:52.730277+0000","last_active":"2021-01-15T19:08:59.185125+0000","last_peered":"2021-01-15T19:08:59.185125+0000","last_clean":"2021-01-15T19:08:59.185125+0000","last_became_active":"2021-01-15T19:08:47.186243+0000","last_became_peered":"2021-01-15T19:08:47.186243+0000","last_unstale":"2021-01-15T19:08:59.185125+0000","last_undegraded":"2021-01-15T19:08:59.185125+0000","last_fullsized":"2021-01-15T19:08:59.185125+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:52.730150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.730150+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"68","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.189520+0000","last_change":"2021-01-15T19:08:35.211897+0000","last_active":"2021-01-15T19:08:59.189520+0000","last_peered":"2021-01-15T19:08:59.189520+0000","last_clean":"2021-01-15T19:08:59.189520+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:08:59.189520+0000","last_undegraded":"2021-01-15T19:08:59.189520+0000","last_fullsized":"2021-01-15T19:08:59.189520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"27","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.198913+0000","last_change":"2021-01-15T19:08:52.205594+0000","last_active":"2021-01-15T19:08:59.198913+0000","last_peered":"2021-01-15T19:08:59.198913+0000","last_clean":"2021-01-15T19:08:59.198913+0000","last_became_active":"2021-01-15T19:08:52.205001+0000","last_became_peered":"2021-01-15T19:08:52.205001+0000","last_unstale":"2021-01-15T19:08:59.198913+0000","last_undegraded":"2021-01-15T19:08:59.198913+0000","last_fullsized":"2021-01-15T19:08:59.198913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"55'64","reported_seq":"206","reported_epoch":"70","state":"active+clean","last_fresh":"2021-01-15T19:08:58.179120+0000","last_change":"2021-01-15T19:08:52.711025+0000","last_active":"2021-01-15T19:08:58.179120+0000","last_peered":"2021-01-15T19:08:58.179120+0000","last_clean":"2021-01-15T19:08:58.179120+0000","last_became_active":"2021-01-15T19:08:41.228824+0000","last_became_peered":"2021-01-15T19:08:41.228824+0000","last_unstale":"2021-01-15T19:08:58.179120+0000","last_undegraded":"2021-01-15T19:08:58.179120+0000","last_fullsized":"2021-01-15T19:08:58.179120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'64","last_scrub_stamp":"2021-01-15T19:08:52.710872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.710872+0000","log_size":64,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":5,"stat_sum":{"num_bytes":4128768,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"1"},{"start":"1c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"84","reported_epoch":"72","state":"active+clean","last_fresh":"2021-01-15T19:09:00.285425+0000","last_change":"2021-01-15T19:08:38.923608+0000","last_active":"2021-01-15T19:09:00.285425+0000","last_peered":"2021-01-15T19:09:00.285425+0000","last_clean":"2021-01-15T19:09:00.285425+0000","last_became_active":"2021-01-15T19:08:35.204089+0000","last_became_peered":"2021-01-15T19:08:35.204089+0000","last_unstale":"2021-01-15T19:09:00.285425+0000","last_undegraded":"2021-01-15T19:09:00.285425+0000","last_fullsized":"2021-01-15T19:09:00.285425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.923498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.923498+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"27","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.189992+0000","last_change":"2021-01-15T19:08:52.211027+0000","last_active":"2021-01-15T19:08:59.189992+0000","last_peered":"2021-01-15T19:08:59.189992+0000","last_clean":"2021-01-15T19:08:59.189992+0000","last_became_active":"2021-01-15T19:08:52.210510+0000","last_became_peered":"2021-01-15T19:08:52.210510+0000","last_unstale":"2021-01-15T19:08:59.189992+0000","last_undegraded":"2021-01-15T19:08:59.189992+0000","last_fullsized":"2021-01-15T19:08:59.189992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"146","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.705801+0000","last_change":"2021-01-15T19:08:59.705801+0000","last_active":"2021-01-15T19:08:59.705801+0000","last_peered":"2021-01-15T19:08:59.705801+0000","last_clean":"2021-01-15T19:08:59.705801+0000","last_became_active":"2021-01-15T19:08:41.228264+0000","last_became_peered":"2021-01-15T19:08:41.228264+0000","last_unstale":"2021-01-15T19:08:59.705801+0000","last_undegraded":"2021-01-15T19:08:59.705801+0000","last_fullsized":"2021-01-15T19:08:59.705801+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:52.822027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.822027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":"72","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.189200+0000","last_change":"2021-01-15T19:08:41.964641+0000","last_active":"2021-01-15T19:08:59.189200+0000","last_peered":"2021-01-15T19:08:59.189200+0000","last_clean":"2021-01-15T19:08:59.189200+0000","last_became_active":"2021-01-15T19:08:35.213894+0000","last_became_peered":"2021-01-15T19:08:35.213894+0000","last_unstale":"2021-01-15T19:08:59.189200+0000","last_undegraded":"2021-01-15T19:08:59.189200+0000","last_fullsized":"2021-01-15T19:08:59.189200+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.964569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.964569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"91","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.185110+0000","last_change":"2021-01-15T19:08:52.199848+0000","last_active":"2021-01-15T19:08:59.185110+0000","last_peered":"2021-01-15T19:08:59.185110+0000","last_clean":"2021-01-15T19:08:59.185110+0000","last_became_active":"2021-01-15T19:08:52.199496+0000","last_became_peered":"2021-01-15T19:08:52.199496+0000","last_unstale":"2021-01-15T19:08:59.185110+0000","last_undegraded":"2021-01-15T19:08:59.185110+0000","last_fullsized":"2021-01-15T19:08:59.185110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'43","reported_seq":"195","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.700232+0000","last_change":"2021-01-15T19:08:59.700232+0000","last_active":"2021-01-15T19:08:59.700232+0000","last_peered":"2021-01-15T19:08:59.700232+0000","last_clean":"2021-01-15T19:08:59.700232+0000","last_became_active":"2021-01-15T19:08:41.230538+0000","last_became_peered":"2021-01-15T19:08:41.230538+0000","last_unstale":"2021-01-15T19:08:59.700232+0000","last_undegraded":"2021-01-15T19:08:59.700232+0000","last_fullsized":"2021-01-15T19:08:59.700232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'31","last_scrub_stamp":"2021-01-15T19:08:49.815451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:49.815451+0000","log_size":43,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3145728,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":"72","reported_epoch":"70","state":"active+clean","last_fresh":"2021-01-15T19:08:58.179501+0000","last_change":"2021-01-15T19:08:39.732649+0000","last_active":"2021-01-15T19:08:58.179501+0000","last_peered":"2021-01-15T19:08:58.179501+0000","last_clean":"2021-01-15T19:08:58.179501+0000","last_became_active":"2021-01-15T19:08:35.204979+0000","last_became_peered":"2021-01-15T19:08:35.204979+0000","last_unstale":"2021-01-15T19:08:58.179501+0000","last_undegraded":"2021-01-15T19:08:58.179501+0000","last_fullsized":"2021-01-15T19:08:58.179501+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:39.732565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:39.732565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"14'2","reported_seq":"91","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.185276+0000","last_change":"2021-01-15T19:08:52.204193+0000","last_active":"2021-01-15T19:08:59.185276+0000","last_peered":"2021-01-15T19:08:59.185276+0000","last_clean":"2021-01-15T19:08:59.185276+0000","last_became_active":"2021-01-15T19:08:52.203692+0000","last_became_peered":"2021-01-15T19:08:52.203692+0000","last_unstale":"2021-01-15T19:08:59.185276+0000","last_undegraded":"2021-01-15T19:08:59.185276+0000","last_fullsized":"2021-01-15T19:08:59.185276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"71'89","reported_seq":"243","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.696612+0000","last_change":"2021-01-15T19:08:59.696612+0000","last_active":"2021-01-15T19:08:59.696612+0000","last_peered":"2021-01-15T19:08:59.696612+0000","last_clean":"2021-01-15T19:08:59.696612+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:08:59.696612+0000","last_undegraded":"2021-01-15T19:08:59.696612+0000","last_fullsized":"2021-01-15T19:08:59.696612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":89,"ondisk_log_size":89,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9338880,"num_objects":9,"num_object_clones":6,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":18,"num_read_kb":2541,"num_write":34,"num_write_kb":10944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.9","version":"0'0","reported_seq":"36","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.190461+0000","last_change":"2021-01-15T19:08:50.981704+0000","last_active":"2021-01-15T19:08:59.190461+0000","last_peered":"2021-01-15T19:08:59.190461+0000","last_clean":"2021-01-15T19:08:59.190461+0000","last_became_active":"2021-01-15T19:08:37.215228+0000","last_became_peered":"2021-01-15T19:08:37.215228+0000","last_unstale":"2021-01-15T19:08:59.190461+0000","last_undegraded":"2021-01-15T19:08:59.190461+0000","last_fullsized":"2021-01-15T19:08:59.190461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:50.981612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:50.981612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"71'50","reported_seq":"207","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.699648+0000","last_change":"2021-01-15T19:08:59.699648+0000","last_active":"2021-01-15T19:08:59.699648+0000","last_peered":"2021-01-15T19:08:59.699648+0000","last_clean":"2021-01-15T19:08:59.699648+0000","last_became_active":"2021-01-15T19:08:41.238249+0000","last_became_peered":"2021-01-15T19:08:41.238249+0000","last_unstale":"2021-01-15T19:08:59.699648+0000","last_undegraded":"2021-01-15T19:08:59.699648+0000","last_fullsized":"2021-01-15T19:08:59.699648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7426048,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":2576,"num_write":21,"num_write_kb":7532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":"36","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.198869+0000","last_change":"2021-01-15T19:08:51.826450+0000","last_active":"2021-01-15T19:08:59.198869+0000","last_peered":"2021-01-15T19:08:59.198869+0000","last_clean":"2021-01-15T19:08:59.198869+0000","last_became_active":"2021-01-15T19:08:37.216976+0000","last_became_peered":"2021-01-15T19:08:37.216976+0000","last_unstale":"2021-01-15T19:08:59.198869+0000","last_undegraded":"2021-01-15T19:08:59.198869+0000","last_fullsized":"2021-01-15T19:08:59.198869+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:51.826360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.826360+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"14'2","reported_seq":"91","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.184744+0000","last_change":"2021-01-15T19:08:52.209378+0000","last_active":"2021-01-15T19:08:59.184744+0000","last_peered":"2021-01-15T19:08:59.184744+0000","last_clean":"2021-01-15T19:08:59.184744+0000","last_became_active":"2021-01-15T19:08:52.204479+0000","last_became_peered":"2021-01-15T19:08:52.204479+0000","last_unstale":"2021-01-15T19:08:59.184744+0000","last_undegraded":"2021-01-15T19:08:59.184744+0000","last_fullsized":"2021-01-15T19:08:59.184744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"71'74","reported_seq":"220","reported_epoch":"72","state":"active+clean","last_fresh":"2021-01-15T19:09:00.291903+0000","last_change":"2021-01-15T19:09:00.207725+0000","last_active":"2021-01-15T19:09:00.291903+0000","last_peered":"2021-01-15T19:09:00.291903+0000","last_clean":"2021-01-15T19:09:00.291903+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:09:00.291903+0000","last_undegraded":"2021-01-15T19:09:00.291903+0000","last_fullsized":"2021-01-15T19:09:00.291903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12173312,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":31,"num_read_kb":23869,"num_write":34,"num_write_kb":11888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.b","version":"0'0","reported_seq":"67","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.189668+0000","last_change":"2021-01-15T19:08:35.210579+0000","last_active":"2021-01-15T19:08:59.189668+0000","last_peered":"2021-01-15T19:08:59.189668+0000","last_clean":"2021-01-15T19:08:59.189668+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:08:59.189668+0000","last_undegraded":"2021-01-15T19:08:59.189668+0000","last_fullsized":"2021-01-15T19:08:59.189668+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:07:55.165623+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"14'2","reported_seq":"95","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.184799+0000","last_change":"2021-01-15T19:08:52.203545+0000","last_active":"2021-01-15T19:08:59.184799+0000","last_peered":"2021-01-15T19:08:59.184799+0000","last_clean":"2021-01-15T19:08:59.184799+0000","last_became_active":"2021-01-15T19:08:52.203041+0000","last_became_peered":"2021-01-15T19:08:52.203041+0000","last_unstale":"2021-01-15T19:08:59.184799+0000","last_undegraded":"2021-01-15T19:08:59.184799+0000","last_fullsized":"2021-01-15T19:08:59.184799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"69'19","reported_seq":"166","reported_epoch":"72","state":"active+clean","last_fresh":"2021-01-15T19:09:00.285559+0000","last_change":"2021-01-15T19:08:59.186751+0000","last_active":"2021-01-15T19:09:00.285559+0000","last_peered":"2021-01-15T19:09:00.285559+0000","last_clean":"2021-01-15T19:09:00.285559+0000","last_became_active":"2021-01-15T19:08:10.231858+0000","last_became_peered":"2021-01-15T19:08:10.231858+0000","last_unstale":"2021-01-15T19:09:00.285559+0000","last_undegraded":"2021-01-15T19:09:00.285559+0000","last_fullsized":"2021-01-15T19:09:00.285559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5324800,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":9822,"num_write":9,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.a","version":"0'0","reported_seq":"71","reported_epoch":"70","state":"active+clean","last_fresh":"2021-01-15T19:08:58.190528+0000","last_change":"2021-01-15T19:08:40.178279+0000","last_active":"2021-01-15T19:08:58.190528+0000","last_peered":"2021-01-15T19:08:58.190528+0000","last_clean":"2021-01-15T19:08:58.190528+0000","last_became_active":"2021-01-15T19:08:37.218600+0000","last_became_peered":"2021-01-15T19:08:37.218600+0000","last_unstale":"2021-01-15T19:08:58.190528+0000","last_undegraded":"2021-01-15T19:08:58.190528+0000","last_fullsized":"2021-01-15T19:08:58.190528+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.178185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.178185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"71'98","reported_seq":"247","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:09:00.215867+0000","last_change":"2021-01-15T19:08:59.704334+0000","last_active":"2021-01-15T19:09:00.215867+0000","last_peered":"2021-01-15T19:09:00.215867+0000","last_clean":"2021-01-15T19:09:00.215867+0000","last_became_active":"2021-01-15T19:08:10.230188+0000","last_became_peered":"2021-01-15T19:08:10.230188+0000","last_unstale":"2021-01-15T19:09:00.215867+0000","last_undegraded":"2021-01-15T19:09:00.215867+0000","last_fullsized":"2021-01-15T19:09:00.215867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16941056,"num_objects":12,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":39,"num_read_kb":17109,"num_write":50,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.d","version":"0'0","reported_seq":"35","reported_epoch":"70","state":"active+clean","last_fresh":"2021-01-15T19:08:58.179584+0000","last_change":"2021-01-15T19:08:41.669593+0000","last_active":"2021-01-15T19:08:58.179584+0000","last_peered":"2021-01-15T19:08:58.179584+0000","last_clean":"2021-01-15T19:08:58.179584+0000","last_became_active":"2021-01-15T19:08:37.216171+0000","last_became_peered":"2021-01-15T19:08:37.216171+0000","last_unstale":"2021-01-15T19:08:58.179584+0000","last_undegraded":"2021-01-15T19:08:58.179584+0000","last_fullsized":"2021-01-15T19:08:58.179584+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.669520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.669520+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"72'55","reported_seq":"202","reported_epoch":"72","state":"active+clean","last_fresh":"2021-01-15T19:09:00.293781+0000","last_change":"2021-01-15T19:08:59.733747+0000","last_active":"2021-01-15T19:09:00.293781+0000","last_peered":"2021-01-15T19:09:00.293781+0000","last_clean":"2021-01-15T19:09:00.293781+0000","last_became_active":"2021-01-15T19:08:10.704086+0000","last_became_peered":"2021-01-15T19:08:10.704086+0000","last_unstale":"2021-01-15T19:09:00.293781+0000","last_undegraded":"2021-01-15T19:09:00.293781+0000","last_fullsized":"2021-01-15T19:09:00.293781+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5128192,"num_objects":9,"num_object_clones":6,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":30,"num_read_kb":3470,"num_write":47,"num_write_kb":4936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.c","version":"0'0","reported_seq":"72","reported_epoch":"70","state":"active+clean","last_fresh":"2021-01-15T19:08:58.190654+0000","last_change":"2021-01-15T19:08:50.105496+0000","last_active":"2021-01-15T19:08:58.190654+0000","last_peered":"2021-01-15T19:08:58.190654+0000","last_clean":"2021-01-15T19:08:58.190654+0000","last_became_active":"2021-01-15T19:08:37.216240+0000","last_became_peered":"2021-01-15T19:08:37.216240+0000","last_unstale":"2021-01-15T19:08:58.190654+0000","last_undegraded":"2021-01-15T19:08:58.190654+0000","last_fullsized":"2021-01-15T19:08:58.190654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:50.105418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:50.105418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"71'28","reported_seq":"169","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:09:00.202840+0000","last_change":"2021-01-15T19:08:59.701250+0000","last_active":"2021-01-15T19:09:00.202840+0000","last_peered":"2021-01-15T19:09:00.202840+0000","last_clean":"2021-01-15T19:09:00.202840+0000","last_became_active":"2021-01-15T19:08:10.233834+0000","last_became_peered":"2021-01-15T19:08:10.233834+0000","last_unstale":"2021-01-15T19:09:00.202840+0000","last_undegraded":"2021-01-15T19:09:00.202840+0000","last_fullsized":"2021-01-15T19:09:00.202840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4456448,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":557,"num_write":13,"num_write_kb":4536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.f","version":"0'0","reported_seq":"72","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.198986+0000","last_change":"2021-01-15T19:08:37.964979+0000","last_active":"2021-01-15T19:08:59.198986+0000","last_peered":"2021-01-15T19:08:59.198986+0000","last_clean":"2021-01-15T19:08:59.198986+0000","last_became_active":"2021-01-15T19:08:37.224346+0000","last_became_peered":"2021-01-15T19:08:37.224346+0000","last_unstale":"2021-01-15T19:08:59.198986+0000","last_undegraded":"2021-01-15T19:08:59.198986+0000","last_fullsized":"2021-01-15T19:08:59.198986+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"71'41","reported_seq":"183","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:09:00.219017+0000","last_change":"2021-01-15T19:08:59.704747+0000","last_active":"2021-01-15T19:09:00.219017+0000","last_peered":"2021-01-15T19:09:00.219017+0000","last_clean":"2021-01-15T19:09:00.219017+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:09:00.219017+0000","last_undegraded":"2021-01-15T19:09:00.219017+0000","last_fullsized":"2021-01-15T19:09:00.219017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5447680,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":2624,"num_write":38,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.e","version":"0'0","reported_seq":"82","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.191436+0000","last_change":"2021-01-15T19:08:41.084609+0000","last_active":"2021-01-15T19:08:59.191436+0000","last_peered":"2021-01-15T19:08:59.191436+0000","last_clean":"2021-01-15T19:08:59.191436+0000","last_became_active":"2021-01-15T19:08:37.216423+0000","last_became_peered":"2021-01-15T19:08:37.216423+0000","last_unstale":"2021-01-15T19:08:59.191436+0000","last_undegraded":"2021-01-15T19:08:59.191436+0000","last_fullsized":"2021-01-15T19:08:59.191436+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.084516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.084516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"69'46","reported_seq":"182","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:09:00.194859+0000","last_change":"2021-01-15T19:09:00.192987+0000","last_active":"2021-01-15T19:09:00.194859+0000","last_peered":"2021-01-15T19:09:00.194859+0000","last_clean":"2021-01-15T19:09:00.194859+0000","last_became_active":"2021-01-15T19:08:46.196080+0000","last_became_peered":"2021-01-15T19:08:46.196080+0000","last_unstale":"2021-01-15T19:09:00.194859+0000","last_undegraded":"2021-01-15T19:09:00.194859+0000","last_fullsized":"2021-01-15T19:09:00.194859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":46,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4341760,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":658,"num_write":23,"num_write_kb":4064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]},{"pgid":"2.10","version":"0'0","reported_seq":"36","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.190431+0000","last_change":"2021-01-15T19:08:42.076013+0000","last_active":"2021-01-15T19:08:59.190431+0000","last_peered":"2021-01-15T19:08:59.190431+0000","last_clean":"2021-01-15T19:08:59.190431+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:08:59.190431+0000","last_undegraded":"2021-01-15T19:08:59.190431+0000","last_fullsized":"2021-01-15T19:08:59.190431+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:08:42.075929+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"36","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.199730+0000","last_change":"2021-01-15T19:08:41.902763+0000","last_active":"2021-01-15T19:08:59.199730+0000","last_peered":"2021-01-15T19:08:59.199730+0000","last_clean":"2021-01-15T19:08:59.199730+0000","last_became_active":"2021-01-15T19:08:37.223920+0000","last_became_peered":"2021-01-15T19:08:37.223920+0000","last_unstale":"2021-01-15T19:08:59.199730+0000","last_undegraded":"2021-01-15T19:08:59.199730+0000","last_fullsized":"2021-01-15T19:08:59.199730+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.902685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.902685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"71'83","reported_seq":"134","reported_epoch":"71","state":"active+clean","last_fresh":"2021-01-15T19:08:59.689739+0000","last_change":"2021-01-15T19:08:59.689739+0000","last_active":"2021-01-15T19:08:59.689739+0000","last_peered":"2021-01-15T19:08:59.689739+0000","last_clean":"2021-01-15T19:08:59.689739+0000","last_became_active":"2021-01-15T19:08:46.202963+0000","last_became_peered":"2021-01-15T19:08:46.202963+0000","last_unstale":"2021-01-15T19:08:59.689739+0000","last_undegraded":"2021-01-15T19:08:59.689739+0000","last_fullsized":"2021-01-15T19:08:59.689739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'67","last_scrub_stamp":"2021-01-15T19:08:49.729161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:49.729161+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4161536,"num_objects":7,"num_object_clones":3,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":5,"num_read_kb":1993,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"10","length":"1"},{"start":"14","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"2"},{"start":"23","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":122056704,"num_objects":105,"num_object_clones":61,"num_object_copies":315,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":3,"num_read":253,"num_read_kb":91166,"num_write":370,"num_write_kb":102620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":12673024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190377984,"data_stored":256757760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1161,"ondisk_log_size":1161,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":22,"acting":22,"num_store_stats":8},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41656,"kb_used_data":33904,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94330184,"statfs":{"total":96636764160,"available":96594108416,"internally_reserved":0,"allocated":34717696,"data_stored":44717925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.121},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.83499999999999996}]},{"osd":1,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.278},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.3959999999999999}]},{"osd":4,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.667}]},{"osd":5,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.22},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1599999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.8340000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":24376,"kb_used_data":15792,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94347464,"statfs":{"total":96636764160,"available":96611803136,"internally_reserved":0,"allocated":16171008,"data_stored":20909465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.2}]},{"osd":1,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.5780000000000001}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":3.2930000000000001,"5min":3.2930000000000001,"15min":3.2930000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.29}]},{"osd":4,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.6990000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.3250000000000002,"5min":3.3250000000000002,"15min":3.3250000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":4.0640000000000001,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.0309999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":3.4289999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.0920000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34672,"kb_used_data":25576,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94337168,"statfs":{"total":96636764160,"available":96601260032,"internally_reserved":0,"allocated":26189824,"data_stored":33369497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.1269999999999998,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8799999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":0.745},{"interface":"front","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":10.584,"5min":10.584,"15min":10.584},"last":1.022}]},{"osd":4,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.077},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.111}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.2230000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"min":{"1min":1.3,"5min":1.3,"15min":1.3},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.5}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28480,"kb_used_data":20408,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94343360,"statfs":{"total":96636764160,"available":96607600640,"internally_reserved":0,"allocated":20897792,"data_stored":25032549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.371}]},{"osd":2,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.24}]},{"osd":3,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.212}]},{"osd":4,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.034}]},{"osd":5,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.46}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.135},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.4099999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.341},{"interface":"front","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.847}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37064,"kb_used_data":23104,"kb_used_omap":0,"kb_used_meta":13952,"kb_avail":94334776,"statfs":{"total":96636764160,"available":96598810624,"internally_reserved":0,"allocated":23658496,"data_stored":36367769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14286848},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.79600000000000004}]},{"osd":1,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.879}]},{"osd":3,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1559999999999999}]},{"osd":4,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.385},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.627},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.466}]},{"osd":7,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.52},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4339999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42288,"kb_used_data":33576,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94329552,"statfs":{"total":96636764160,"available":96593461248,"internally_reserved":0,"allocated":34381824,"data_stored":44336997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":1.161},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.002}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.137},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.4330000000000001}]},{"osd":4,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.569}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.306},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.1879999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.212}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44916,"kb_used_data":29484,"kb_used_omap":0,"kb_used_meta":15424,"kb_avail":94326924,"statfs":{"total":96636764160,"available":96590770176,"internally_reserved":0,"allocated":30191616,"data_stored":44125593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15794176},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.252}]},{"osd":1,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.5529999999999999}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.423}]},{"osd":3,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":2.758},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":2.5390000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.087},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":2.0059999999999998}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.448},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.3420000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":2.2440000000000002},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":2.085}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15624,"kb_used_data":9344,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":94356216,"statfs":{"total":96636764160,"available":96620765184,"internally_reserved":0,"allocated":9568256,"data_stored":9858457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.2450000000000001}]},{"osd":1,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.073}]},{"osd":2,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.28}]},{"osd":3,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.363}]},{"osd":4,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.345}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.042}]},{"osd":7,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.4470000000000001}]}]}]}} 2021-01-15T19:09:04.025 INFO:tasks.thrashosds.thrasher:Setting 2.b to [1, 3, 4, 0] 2021-01-15T19:09:04.025 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.b', '1', '3', '4', '0'] 2021-01-15T19:09:04.025 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.b 1 3 4 0 2021-01-15T19:09:04.060 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:04.055+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:04.135 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.845634869000001, 2021-01-15T19:09:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.138 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.913872103000003, 2021-01-15T19:09:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.167 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.911584062000003, 2021-01-15T19:09:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.908994436999997, 2021-01-15T19:09:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.908086159, 2021-01-15T19:09:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:04.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:04.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:04.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:04.268 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.939402252000001, 2021-01-15T19:09:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:04.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:04.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:04.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:04.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:04.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:04.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:04.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:04.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:04.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:04.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:04.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:04.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:04.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:04.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:04.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:04.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:04.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:04.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:04.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:04.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:04.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:04.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.303 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.303 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:04.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:04.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.939252515, 2021-01-15T19:09:04.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:04.306 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.307 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.307 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:04.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.308 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:04.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:04.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:04.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:04.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:04.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:04.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:04.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:04.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:04.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:04.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:04.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:04.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:04.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:04.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:04.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:04.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:04.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:04.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:04.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:04.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:04.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:04.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:04.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:04.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:04.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:04.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:04.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:04.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:04.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:04.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:04.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:04.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.326 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:04.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:04.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.939100617000001, 2021-01-15T19:09:04.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:04.327 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.327 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.328 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:04.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.328 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:04.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:04.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:04.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:04.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:04.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:04.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:04.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:04.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:04.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:04.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:04.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:04.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:04.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:04.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:04.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:04.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:04.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:04.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:04.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:04.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:04.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:04.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:04.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:04.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:04.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:04.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:04.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.346 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:04.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:04.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.938975305, 2021-01-15T19:09:04.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:04.348 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.349 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:04.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.349 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:04.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:04.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:04.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:04.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:04.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:04.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:04.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:04.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:04.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:04.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:04.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:04.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:04.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:04.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:04.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:04.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:04.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:04.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:04.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:04.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:04.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:04.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:04.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:04.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:04.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:04.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:04.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:04.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:04.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:04.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.374 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.375 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:04.375 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:04.376 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.91814273, 2021-01-15T19:09:04.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:04.376 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.377 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.377 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.377 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.377 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.378 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:04.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.379 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:04.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:04.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:04.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:04.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:04.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:04.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:04.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:04.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:04.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:04.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:04.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:04.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:04.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:04.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:04.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:04.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:04.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:04.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:04.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:04.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:04.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:04.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:04.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.420 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.423 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:04.423 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:04.423 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.911849398000001, 2021-01-15T19:09:04.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:04.424 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.424 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.425 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:04.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.425 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:04.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:04.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:04.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:04.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:04.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:04.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:04.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:04.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:04.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:04.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:04.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:04.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:04.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:04.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:04.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:04.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:04.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:04.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:04.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:04.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:04.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:04.436 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.437 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:04.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:04.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.949789885000001, 2021-01-15T19:09:04.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:04.438 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.438 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.439 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:04.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.440 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:04.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:04.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:04.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:04.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:04.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:04.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:04.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:04.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:04.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:04.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:04.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:04.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:04.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:04.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:04.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:04.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:04.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:04.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:04.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:04.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:04.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:04.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:04.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:04.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:04.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:04.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.454 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:04.455 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:04.455 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.887417905, 2021-01-15T19:09:04.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:04.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:04.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.457 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:04.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:04.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:04.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:04.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:04.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:04.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:04.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:04.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:04.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.884115403999999, 2021-01-15T19:09:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:04.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:04.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:04.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:04.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:04.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:04.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:04.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:04.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:04.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:04.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.882793538000001, 2021-01-15T19:09:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.882423285000002, 2021-01-15T19:09:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.882088891999999, 2021-01-15T19:09:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.940827633, 2021-01-15T19:09:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.94846476099999999, 2021-01-15T19:09:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.91847055700000002, 2021-01-15T19:09:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:04.638 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:04.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:04.646 INFO:tasks.rados.rados.0.smithi001.stdout:283: done (2 left) 2021-01-15T19:09:04.646 INFO:tasks.rados.rados.0.smithi001.stdout:284: read oid 2 snap 25 2021-01-15T19:09:04.646 INFO:tasks.rados.rados.0.smithi001.stdout:284: expect (ObjNum 115 snap 25 seq_num 115) 2021-01-15T19:09:04.647 INFO:tasks.rados.rados.0.smithi001.stdout:285: read oid 15 snap -1 2021-01-15T19:09:04.647 INFO:tasks.rados.rados.0.smithi001.stdout:285: expect (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:09:04.647 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 12 current snap is 30 2021-01-15T19:09:04.647 INFO:tasks.rados.rados.0.smithi001.stdout:280: finishing write tid 6 to smithi00140323-46 2021-01-15T19:09:04.647 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 46 v 41 (ObjNum 126 snap 30 seq_num 126) dirty exists 2021-01-15T19:09:04.648 INFO:tasks.rados.rados.0.smithi001.stdout:280: left oid 46 (ObjNum 126 snap 30 seq_num 126) 2021-01-15T19:09:04.648 INFO:tasks.rados.rados.0.smithi001.stdout:282: expect (ObjNum 76 snap 12 seq_num 76) 2021-01-15T19:09:04.648 INFO:tasks.rados.rados.0.smithi001.stdout:285: expect (ObjNum 51 snap 1 seq_num 51) 2021-01-15T19:09:04.648 INFO:tasks.rados.rados.0.smithi001.stdout:284: expect (ObjNum 115 snap 25 seq_num 115) 2021-01-15T19:09:04.649 INFO:tasks.rados.rados.0.smithi001.stdout:286: seq_num 127 ranges {1990656=532480,2523136=507904,3031040=778240,3809280=319488} 2021-01-15T19:09:04.649 INFO:tasks.rados.rados.0.smithi001.stdout:286: writing smithi00140323-12 from 1990656 to 2523136 tid 1 2021-01-15T19:09:04.649 INFO:tasks.rados.rados.0.smithi001.stdout:286: writing smithi00140323-12 from 2523136 to 3031040 tid 2 2021-01-15T19:09:04.649 INFO:tasks.rados.rados.0.smithi001.stdout:286: writing smithi00140323-12 from 3031040 to 3809280 tid 3 2021-01-15T19:09:04.650 INFO:tasks.rados.rados.0.smithi001.stdout:286: writing smithi00140323-12 from 3809280 to 4128768 tid 4 2021-01-15T19:09:04.650 INFO:tasks.rados.rados.0.smithi001.stdout:280: done (4 left) 2021-01-15T19:09:04.650 INFO:tasks.rados.rados.0.smithi001.stdout:282: done (3 left) 2021-01-15T19:09:04.650 INFO:tasks.rados.rados.0.smithi001.stdout:284: done (2 left) 2021-01-15T19:09:04.650 INFO:tasks.rados.rados.0.smithi001.stdout:285: done (1 left) 2021-01-15T19:09:04.651 INFO:tasks.rados.rados.0.smithi001.stdout:287: snap_create 2021-01-15T19:09:04.651 INFO:tasks.rados.rados.0.smithi001.stdout:286: finishing write tid 1 to smithi00140323-12 2021-01-15T19:09:04.651 INFO:tasks.rados.rados.0.smithi001.stdout:286: finishing write tid 2 to smithi00140323-12 2021-01-15T19:09:04.651 INFO:tasks.rados.rados.0.smithi001.stdout:286: finishing write tid 3 to smithi00140323-12 2021-01-15T19:09:04.652 INFO:tasks.rados.rados.0.smithi001.stdout:286: finishing write tid 4 to smithi00140323-12 2021-01-15T19:09:04.652 INFO:tasks.rados.rados.0.smithi001.stdout:286: finishing write tid 5 to smithi00140323-12 2021-01-15T19:09:04.652 INFO:tasks.rados.rados.0.smithi001.stdout:286: finishing write tid 6 to smithi00140323-12 2021-01-15T19:09:04.653 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 34 (ObjNum 127 snap 30 seq_num 127) dirty exists 2021-01-15T19:09:04.653 INFO:tasks.rados.rados.0.smithi001.stdout:286: left oid 12 (ObjNum 127 snap 30 seq_num 127) 2021-01-15T19:09:04.653 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:09:04.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:09:04.655 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:04.639+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:04.741 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:04.740+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:04.816 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:04.818 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:04.827 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:09:04.832 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:04.844 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:04.842+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:04.854 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:04.872 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:04.893 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:04.915 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:04.941 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:04.945 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:04.943+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:04.958 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:04.959 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:04.966 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:09:04.976 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:04.979 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.980 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.980 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.981 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.981 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.982 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.982 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:09:04.983 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:05.048 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:05.045+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:05.096 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:05.097 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:05.107 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:05.148 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:05.147+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:05.249 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:05.248+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:05.274 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:05.275 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:05.287 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:05.308 INFO:tasks.rados.rados.0.smithi001.stdout:286: done (1 left) 2021-01-15T19:09:05.309 INFO:tasks.rados.rados.0.smithi001.stdout:287: done (0 left) 2021-01-15T19:09:05.309 INFO:tasks.rados.rados.0.smithi001.stdout:288: delete oid 8 current snap is 31 2021-01-15T19:09:05.310 INFO:teuthology.orchestra.run.smithi001.stderr:set 2.b pg_upmap_items mapping to [1->3,4->0] 2021-01-15T19:09:05.318 INFO:tasks.rados.rados.0.smithi001.stdout:288: done (0 left) 2021-01-15T19:09:05.319 INFO:tasks.rados.rados.0.smithi001.stdout:289: delete oid 46 current snap is 31 2021-01-15T19:09:05.327 INFO:tasks.rados.rados.0.smithi001.stdout:289: done (0 left) 2021-01-15T19:09:05.328 INFO:tasks.rados.rados.0.smithi001.stdout:290: copy_from oid 10 from oid 23 current snap is 31 2021-01-15T19:09:05.328 INFO:tasks.rados.rados.0.smithi001.stdout:291: read oid 5 snap -1 2021-01-15T19:09:05.329 INFO:tasks.rados.rados.0.smithi001.stdout:291: expect (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:09:05.329 INFO:tasks.rados.rados.0.smithi001.stdout:292: read oid 3 snap -1 2021-01-15T19:09:05.329 INFO:tasks.rados.rados.0.smithi001.stdout:292: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:05.330 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 21 current snap is 31 2021-01-15T19:09:05.330 INFO:tasks.rados.rados.0.smithi001.stdout:293: seq_num 130 ranges {761856=581632,1343488=614400,1957888=598016,2555904=393216} 2021-01-15T19:09:05.333 INFO:tasks.rados.rados.0.smithi001.stdout:293: writing smithi00140323-21 from 761856 to 1343488 tid 1 2021-01-15T19:09:05.339 INFO:tasks.rados.rados.0.smithi001.stdout:293: writing smithi00140323-21 from 1343488 to 1957888 tid 2 2021-01-15T19:09:05.345 INFO:tasks.rados.rados.0.smithi001.stdout:293: writing smithi00140323-21 from 1957888 to 2555904 tid 3 2021-01-15T19:09:05.345 INFO:tasks.rados.rados.0.smithi001.stdout:293: writing smithi00140323-21 from 2555904 to 2949120 tid 4 2021-01-15T19:09:05.345 INFO:tasks.rados.rados.0.smithi001.stdout:294: rollback oid 4 current snap is 31 2021-01-15T19:09:05.346 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 4 to 14 2021-01-15T19:09:05.346 INFO:tasks.rados.rados.0.smithi001.stdout:291: expect (ObjNum 4 snap 0 seq_num 4) 2021-01-15T19:09:05.347 INFO:tasks.rados.rados.0.smithi001.stdout:290: finishing copy_from to smithi00140323-10 2021-01-15T19:09:05.347 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2021-01-15T19:09:05.347 INFO:tasks.rados.rados.0.smithi001.stdout:290: finishing copy_from racing read to smithi00140323-10 2021-01-15T19:09:05.347 INFO:tasks.rados.rados.0.smithi001.stdout:290: done (4 left) 2021-01-15T19:09:05.347 INFO:tasks.rados.rados.0.smithi001.stdout:291: done (3 left) 2021-01-15T19:09:05.348 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 8 current snap is 31 2021-01-15T19:09:05.348 INFO:tasks.rados.rados.0.smithi001.stdout:293: finishing write tid 1 to smithi00140323-21 2021-01-15T19:09:05.348 INFO:tasks.rados.rados.0.smithi001.stdout:295: seq_num 131 ranges {0=598016,598016=499712,1097728=753664,1851392=352256} 2021-01-15T19:09:05.350 INFO:tasks.rados.rados.0.smithi001.stdout:295: writing smithi00140323-8 from 0 to 598016 tid 1 2021-01-15T19:09:05.351 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:05.349+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:05.354 INFO:tasks.rados.rados.0.smithi001.stdout:295: writing smithi00140323-8 from 598016 to 1097728 tid 2 2021-01-15T19:09:05.358 INFO:tasks.rados.rados.0.smithi001.stdout:295: writing smithi00140323-8 from 1097728 to 1851392 tid 3 2021-01-15T19:09:05.360 INFO:tasks.rados.rados.0.smithi001.stdout:295: writing smithi00140323-8 from 1851392 to 2203648 tid 4 2021-01-15T19:09:05.361 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 35 current snap is 31 2021-01-15T19:09:05.361 INFO:tasks.rados.rados.0.smithi001.stdout:292: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:05.425 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:05.426 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:05.439 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:05.452 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:05.451+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:05.468 INFO:tasks.rados.rados.0.smithi001.stdout:293: finishing write tid 2 to smithi00140323-21 2021-01-15T19:09:05.469 INFO:tasks.rados.rados.0.smithi001.stdout:294: finishing rollback tid 0 to smithi00140323-4 2021-01-15T19:09:05.469 INFO:tasks.rados.rados.0.smithi001.stdout:294: finishing rollback tid 1 to smithi00140323-4 2021-01-15T19:09:05.469 INFO:tasks.rados.rados.0.smithi001.stdout:294: finishing rollback tid 2 to smithi00140323-4 2021-01-15T19:09:05.470 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 45 (ObjNum 3 snap 0 seq_num 3) dirty exists 2021-01-15T19:09:05.471 INFO:tasks.rados.rados.0.smithi001.stdout:293: finishing write tid 3 to smithi00140323-21 2021-01-15T19:09:05.472 INFO:tasks.rados.rados.0.smithi001.stdout:293: finishing write tid 4 to smithi00140323-21 2021-01-15T19:09:05.472 INFO:tasks.rados.rados.0.smithi001.stdout:293: finishing write tid 5 to smithi00140323-21 2021-01-15T19:09:05.473 INFO:tasks.rados.rados.0.smithi001.stdout:293: finishing write tid 6 to smithi00140323-21 2021-01-15T19:09:05.473 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 49 (ObjNum 130 snap 31 seq_num 130) dirty exists 2021-01-15T19:09:05.473 INFO:tasks.rados.rados.0.smithi001.stdout:293: left oid 21 (ObjNum 130 snap 31 seq_num 130) 2021-01-15T19:09:05.474 INFO:tasks.rados.rados.0.smithi001.stdout:295: finishing write tid 1 to smithi00140323-8 2021-01-15T19:09:05.474 INFO:tasks.rados.rados.0.smithi001.stdout:295: finishing write tid 2 to smithi00140323-8 2021-01-15T19:09:05.475 INFO:tasks.rados.rados.0.smithi001.stdout:295: finishing write tid 3 to smithi00140323-8 2021-01-15T19:09:05.475 INFO:tasks.rados.rados.0.smithi001.stdout:295: finishing write tid 4 to smithi00140323-8 2021-01-15T19:09:05.476 INFO:tasks.rados.rados.0.smithi001.stdout:295: finishing write tid 5 to smithi00140323-8 2021-01-15T19:09:05.476 INFO:tasks.rados.rados.0.smithi001.stdout:295: finishing write tid 6 to smithi00140323-8 2021-01-15T19:09:05.476 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 59 (ObjNum 131 snap 31 seq_num 131) dirty exists 2021-01-15T19:09:05.486 INFO:tasks.rados.rados.0.smithi001.stdout:295: left oid 8 (ObjNum 131 snap 31 seq_num 131) 2021-01-15T19:09:05.486 INFO:tasks.rados.rados.0.smithi001.stdout:296: seq_num 132 ranges {1556480=614400,2170880=786432,2957312=565248,3522560=253952} 2021-01-15T19:09:05.487 INFO:tasks.rados.rados.0.smithi001.stdout:296: writing smithi00140323-35 from 1556480 to 2170880 tid 1 2021-01-15T19:09:05.487 INFO:tasks.rados.rados.0.smithi001.stdout:296: writing smithi00140323-35 from 2170880 to 2957312 tid 2 2021-01-15T19:09:05.487 INFO:tasks.rados.rados.0.smithi001.stdout:296: writing smithi00140323-35 from 2957312 to 3522560 tid 3 2021-01-15T19:09:05.491 INFO:tasks.rados.rados.0.smithi001.stdout:296: writing smithi00140323-35 from 3522560 to 3776512 tid 4 2021-01-15T19:09:05.491 INFO:tasks.rados.rados.0.smithi001.stdout:292: done (4 left) 2021-01-15T19:09:05.492 INFO:tasks.rados.rados.0.smithi001.stdout:293: done (3 left) 2021-01-15T19:09:05.492 INFO:tasks.rados.rados.0.smithi001.stdout:294: done (2 left) 2021-01-15T19:09:05.492 INFO:tasks.rados.rados.0.smithi001.stdout:295: done (1 left) 2021-01-15T19:09:05.493 INFO:tasks.rados.rados.0.smithi001.stdout:297: snap_remove snap 29 2021-01-15T19:09:05.554 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:05.553+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:05.569 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:05.570 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:05.579 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:05.657 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:05.655+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:05.714 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:05.715 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:05.727 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:05.759 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:05.758+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:05.861 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:05.861 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:05.862 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:05.860+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:05.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:05.962 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:05.961+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:06.006 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:06.006 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:06.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:06.070 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:06.066+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:06.154 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:06.155 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:06.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:06.176 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:06.174+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:06.280 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:06.278+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:06.312 INFO:tasks.rados.rados.0.smithi001.stdout:297: done (1 left) 2021-01-15T19:09:06.313 INFO:tasks.rados.rados.0.smithi001.stdout:298: delete oid 10 current snap is 31 2021-01-15T19:09:06.313 INFO:tasks.rados.rados.0.smithi001.stdout:296: finishing write tid 1 to smithi00140323-35 2021-01-15T19:09:06.313 INFO:tasks.rados.rados.0.smithi001.stdout:296: finishing write tid 2 to smithi00140323-35 2021-01-15T19:09:06.313 INFO:tasks.rados.rados.0.smithi001.stdout:296: finishing write tid 3 to smithi00140323-35 2021-01-15T19:09:06.314 INFO:tasks.rados.rados.0.smithi001.stdout:296: finishing write tid 4 to smithi00140323-35 2021-01-15T19:09:06.314 INFO:tasks.rados.rados.0.smithi001.stdout:296: finishing write tid 5 to smithi00140323-35 2021-01-15T19:09:06.314 INFO:tasks.rados.rados.0.smithi001.stdout:296: finishing write tid 6 to smithi00140323-35 2021-01-15T19:09:06.314 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 80 (ObjNum 132 snap 31 seq_num 132) dirty exists 2021-01-15T19:09:06.314 INFO:tasks.rados.rados.0.smithi001.stdout:296: left oid 35 (ObjNum 132 snap 31 seq_num 132) 2021-01-15T19:09:06.317 INFO:tasks.rados.rados.0.smithi001.stdout:296: done (1 left) 2021-01-15T19:09:06.318 INFO:tasks.rados.rados.0.smithi001.stdout:298: done (0 left) 2021-01-15T19:09:06.318 INFO:tasks.rados.rados.0.smithi001.stdout:299: delete oid 14 current snap is 31 2021-01-15T19:09:06.326 INFO:tasks.rados.rados.0.smithi001.stdout:299: done (0 left) 2021-01-15T19:09:06.327 INFO:tasks.rados.rados.0.smithi001.stdout:300: snap_create 2021-01-15T19:09:06.350 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:06.351 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:06.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:06.382 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:06.379+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:06.481 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:06.480+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:06.535 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:06.535 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:06.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:06.596 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:06.594+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:06.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:09:06.684 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:06.684 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:06.694 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:06.686+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:06.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:06.791 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:06.787+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:06.877 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:06.878 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:06.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:06.892 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:06.890+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:06.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:06.998 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:06.994+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:07.024 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:07.025 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:07.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:07.099 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:07.095+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:07.151 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:07.163 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:07.185 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:07.199 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:07.197+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:07.204 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:07.222 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:07.223 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:07.223 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:07.223 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:07.227 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:07.230 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:07.251 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:07.275 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:07.303 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.304 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.304 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.304 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.304 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:07.306 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:07.302+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:07.311 INFO:tasks.rados.rados.0.smithi001.stdout:300: done (0 left) 2021-01-15T19:09:07.311 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 34 current snap is 32 2021-01-15T19:09:07.311 INFO:tasks.rados.rados.0.smithi001.stdout:301: seq_num 135 ranges {2703360=663552,3366912=434176,3801088=786432,4587520=385024} 2021-01-15T19:09:07.313 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:09:07.330 INFO:tasks.rados.rados.0.smithi001.stdout:301: writing smithi00140323-34 from 2703360 to 3366912 tid 1 2021-01-15T19:09:07.333 INFO:tasks.rados.rados.0.smithi001.stdout:301: writing smithi00140323-34 from 3366912 to 3801088 tid 2 2021-01-15T19:09:07.339 INFO:tasks.rados.rados.0.smithi001.stdout:301: writing smithi00140323-34 from 3801088 to 4587520 tid 3 2021-01-15T19:09:07.342 INFO:tasks.rados.rados.0.smithi001.stdout:301: writing smithi00140323-34 from 4587520 to 4972544 tid 4 2021-01-15T19:09:07.343 INFO:tasks.rados.rados.0.smithi001.stdout:302: copy_from oid 36 from oid 3 current snap is 32 2021-01-15T19:09:07.343 INFO:tasks.rados.rados.0.smithi001.stdout:303: delete oid 45 current snap is 32 2021-01-15T19:09:07.345 INFO:tasks.rados.rados.0.smithi001.stdout:301: finishing write tid 1 to smithi00140323-34 2021-01-15T19:09:07.346 INFO:tasks.rados.rados.0.smithi001.stdout:301: finishing write tid 2 to smithi00140323-34 2021-01-15T19:09:07.356 INFO:tasks.rados.rados.0.smithi001.stdout:301: finishing write tid 3 to smithi00140323-34 2021-01-15T19:09:07.356 INFO:tasks.rados.rados.0.smithi001.stdout:301: finishing write tid 4 to smithi00140323-34 2021-01-15T19:09:07.357 INFO:tasks.rados.rados.0.smithi001.stdout:301: finishing write tid 5 to smithi00140323-34 2021-01-15T19:09:07.357 INFO:tasks.rados.rados.0.smithi001.stdout:303: done (2 left) 2021-01-15T19:09:07.357 INFO:tasks.rados.rados.0.smithi001.stdout:304: read oid 29 snap -1 2021-01-15T19:09:07.357 INFO:tasks.rados.rados.0.smithi001.stdout:304: expect (ObjNum 122 snap 27 seq_num 122) 2021-01-15T19:09:07.358 INFO:tasks.rados.rados.0.smithi001.stdout:305: read oid 24 snap -1 2021-01-15T19:09:07.358 INFO:tasks.rados.rados.0.smithi001.stdout:305: expect (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:09:07.358 INFO:tasks.rados.rados.0.smithi001.stdout:306: snap_remove snap 24 2021-01-15T19:09:07.402 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:07.403 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:07.404 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:07.403+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:07.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:07.507 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:07.505+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:07.578 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:07.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:07.579 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.326733694000001, 2021-01-15T19:09:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.580 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:07.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:07.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:07.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:07.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.326542750000002, 2021-01-15T19:09:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.611 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:07.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:07.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:07.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:07.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:07.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:07.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:07.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:07.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.630 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.387590653, 2021-01-15T19:09:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:07.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:07.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:07.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:07.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:07.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:07.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:07.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:07.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:07.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:07.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:07.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:07.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:07.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.651 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:07.651 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:07.651 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.384870192000001, 2021-01-15T19:09:07.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:07.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:07.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.653 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:07.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:07.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:07.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:07.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:07.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:07.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:07.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:07.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:07.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:07.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:07.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:07.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:07.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:07.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.384757613000001, 2021-01-15T19:09:07.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:07.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.674 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:07.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:07.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:07.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:07.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:07.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:07.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:07.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:07.690 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.691 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.692 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:07.692 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:07.692 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.384627709, 2021-01-15T19:09:07.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:07.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:07.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.694 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:07.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.695 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:07.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:07.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:07.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:07.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:07.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:07.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:07.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:07.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:07.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:07.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:07.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:07.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:07.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:07.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:07.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:07.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:07.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:07.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:07.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:07.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:07.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:07.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:07.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:07.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:07.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:07.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:07.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.714 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.714 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:07.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.361422439999998, 2021-01-15T19:09:07.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:07.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.716 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:07.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.717 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:07.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:07.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:07.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:07.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:07.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:07.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:07.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:07.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:07.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:07.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:07.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:07.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:07.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:07.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:07.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:07.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:07.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:07.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:07.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:07.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:07.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:07.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:07.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.361304241999999, 2021-01-15T19:09:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:07.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:07.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.739 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:07.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:07.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:07.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:07.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:07.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:07.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:07.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:07.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:07.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:07.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:07.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:07.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:07.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:07.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:07.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:07.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:07.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:07.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:07.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:07.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:07.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:07.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:07.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.756 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.757 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:07.757 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.361191552000001, 2021-01-15T19:09:07.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:07.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:07.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.759 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:07.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:07.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:07.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:07.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:07.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:07.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:07.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:07.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:07.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:07.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:07.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:07.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:07.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:07.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:07.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:07.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:07.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:07.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:07.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:07.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:07.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:07.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:07.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:07.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:07.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.355589802000001, 2021-01-15T19:09:07.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:07.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:07.777 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:07.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:07.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:07.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:07.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:07.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:07.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:07.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:07.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:07.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:07.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:07.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:07.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:07.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:07.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:07.787 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:07.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.347121078000001, 2021-01-15T19:09:07.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:07.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:07.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:07.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:07.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:07.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:07.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:07.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:07.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:07.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:07.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:07.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:07.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:07.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:07.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:07.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:07.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:07.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:07.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:07.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:07.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:07.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:07.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.807 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.808 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:07.809 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.342602495000001, 2021-01-15T19:09:07.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:07.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:07.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.811 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:07.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:07.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:07.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:07.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:07.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:07.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:07.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:07.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:07.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:07.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:07.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:07.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:07.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:07.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:07.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:07.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:07.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:07.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:07.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:07.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:07.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:07.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:07.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.826 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.826 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:07.826 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.338195398, 2021-01-15T19:09:07.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:07.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.828 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:07.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.829 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:07.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:07.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:07.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:07.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:07.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:07.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:07.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:07.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:07.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:07.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:07.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:07.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:07.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:07.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:07.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:07.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:07.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:07.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:07.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:07.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:07.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:07.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:07.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:07.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:07.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.337713271999998, 2021-01-15T19:09:07.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:07.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:07.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.847 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:07.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:07.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:07.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:07.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:07.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:07.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:07.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:07.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:07.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:07.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:07.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:07.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:07.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:07.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:07.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:07.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:07.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:07.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:07.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:07.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:07.861 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:07.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:07.863 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.337584725999999, 2021-01-15T19:09:07.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:07.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:07.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.864 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:07.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.865 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:07.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:07.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:07.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:07.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:07.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:07.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:07.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:07.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:07.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:07.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:07.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:07.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:07.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:07.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:07.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:07.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:07.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:07.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:07.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:07.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:07.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:07.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.877 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.878 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:07.878 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:07.879 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.329135586, 2021-01-15T19:09:07.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:07.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:07.879 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:07.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:07.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:07.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:07.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:07.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:07.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:07.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:07.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:07.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:07.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:07.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:07.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:07.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:07.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:07.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.889 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:07.890 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:07.890 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.3704606479999999, 2021-01-15T19:09:07.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:07.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:07.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.892 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:07.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:07.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:07.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:07.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:07.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:07.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:07.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:07.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:07.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:07.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:07.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:07.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:07.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:07.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:07.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:07.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:07.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:07.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:07.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:07.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:07.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:07.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:07.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:07.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:07.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:07.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.911 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.912 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:697 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~704512] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:07.912 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.209222+0000", 2021-01-15T19:09:07.913 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.3664049420000008, 2021-01-15T19:09:07.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019373363000000001, 2021-01-15T19:09:07.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:07.913 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 697 2021-01-15T19:09:07.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.914 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.209222+0000", 2021-01-15T19:09:07.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.209222+0000", 2021-01-15T19:09:07.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00054519900000000001 2021-01-15T19:09:07.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.209767+0000", 2021-01-15T19:09:07.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0380999999999999e-05 2021-01-15T19:09:07.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.209777+0000", 2021-01-15T19:09:07.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6609e-05 2021-01-15T19:09:07.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.209794+0000", 2021-01-15T19:09:07.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8409999999999995e-06 2021-01-15T19:09:07.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.209804+0000", 2021-01-15T19:09:07.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0077996130000000004 2021-01-15T19:09:07.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.217603+0000", 2021-01-15T19:09:07.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3602000000000003e-05 2021-01-15T19:09:07.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.217697+0000", 2021-01-15T19:09:07.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000887166 2021-01-15T19:09:07.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.218584+0000", 2021-01-15T19:09:07.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0098426209999999993 2021-01-15T19:09:07.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228427+0000", 2021-01-15T19:09:07.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117953 2021-01-15T19:09:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228545+0000", 2021-01-15T19:09:07.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0377999999999998e-05 2021-01-15T19:09:07.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228595+0000", 2021-01-15T19:09:07.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019373363000000001 2021-01-15T19:09:07.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.928 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:07.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:07.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.2462759659999998, 2021-01-15T19:09:07.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:07.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:07.930 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:07.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:07.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:07.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:07.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:07.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:07.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:07.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:07.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:07.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:07.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:07.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:07.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:07.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:07.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:07.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:07.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:07.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:07.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:07.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:07.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:07.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.940 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.941 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:07.941 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:07.941 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.3849146779999999, 2021-01-15T19:09:07.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:07.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:07.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:07.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:07.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:07.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:07.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:07.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.943 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:07.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:07.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:07.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:07.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:07.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:07.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:07.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:07.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:07.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:07.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:07.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:07.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:07.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:07.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:07.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:07.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:07.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:07.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:07.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:07.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:07.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:07.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:07.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:07.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:07.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:07.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:07.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:07.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:07.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:07.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:07.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:07.973 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:07.974 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:07.973+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:07.975 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:08.082 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:08.075+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:08.120 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:08.121 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:08.121 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:08.121 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:08.129 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:08.189 INFO:tasks.rados.rados.0.smithi001.stdout:306: done (4 left) 2021-01-15T19:09:08.189 INFO:tasks.rados.rados.0.smithi001.stdout:307: read oid 39 snap 23 2021-01-15T19:09:08.190 INFO:tasks.rados.rados.0.smithi001.stdout:307: expect (ObjNum 99 snap 16 seq_num 99) 2021-01-15T19:09:08.190 INFO:tasks.rados.rados.0.smithi001.stdout:308: read oid 22 snap -1 2021-01-15T19:09:08.190 INFO:tasks.rados.rados.0.smithi001.stdout:308: expect (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:09:08.190 INFO:tasks.rados.rados.0.smithi001.stdout:309: rollback oid 25 current snap is 32 2021-01-15T19:09:08.190 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 25 to 13 2021-01-15T19:09:08.191 INFO:tasks.rados.rados.0.smithi001.stdout:301: finishing write tid 6 to smithi00140323-34 2021-01-15T19:09:08.191 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 34 v 110 (ObjNum 135 snap 32 seq_num 135) dirty exists 2021-01-15T19:09:08.191 INFO:tasks.rados.rados.0.smithi001.stdout:301: left oid 34 (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:09:08.191 INFO:tasks.rados.rados.0.smithi001.stdout:301: done (6 left) 2021-01-15T19:09:08.191 INFO:tasks.rados.rados.0.smithi001.stdout:310: delete oid 35 current snap is 32 2021-01-15T19:09:08.192 INFO:tasks.rados.rados.0.smithi001.stdout:305: expect (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:09:08.192 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:08.181+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:08.216 INFO:tasks.rados.rados.0.smithi001.stdout:304: expect (ObjNum 122 snap 27 seq_num 122) 2021-01-15T19:09:08.268 INFO:tasks.rados.rados.0.smithi001.stdout:302: finishing copy_from to smithi00140323-36 2021-01-15T19:09:08.268 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:08.269 INFO:tasks.rados.rados.0.smithi001.stdout:302: finishing copy_from racing read to smithi00140323-36 2021-01-15T19:09:08.269 INFO:tasks.rados.rados.0.smithi001.stdout:308: expect (ObjNum 40 snap 0 seq_num 40) 2021-01-15T19:09:08.274 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:08.274 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:08.274 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:08.275 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:08.275 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:08.288 INFO:tasks.rados.rados.0.smithi001.stdout:309: finishing rollback tid 0 to smithi00140323-25 2021-01-15T19:09:08.288 INFO:tasks.rados.rados.0.smithi001.stdout:309: finishing rollback tid 1 to smithi00140323-25 2021-01-15T19:09:08.288 INFO:tasks.rados.rados.0.smithi001.stdout:309: finishing rollback tid 2 to smithi00140323-25 2021-01-15T19:09:08.289 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 95 (ObjNum 24 snap 0 seq_num 24) dirty exists 2021-01-15T19:09:08.289 INFO:tasks.rados.rados.0.smithi001.stdout:307: expect (ObjNum 99 snap 16 seq_num 99) 2021-01-15T19:09:08.289 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:08.287+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:08.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:08.388 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:08.387+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:08.458 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:08.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:365 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [setxattr _header (57) in=7b] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:08.460 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:09:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.263769953999997, 2021-01-15T19:09:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014071523000000001, 2021-01-15T19:09:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.462 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 365 2021-01-15T19:09:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:09:08.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191841+0000", 2021-01-15T19:09:08.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5833000000000002e-05 2021-01-15T19:09:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191867+0000", 2021-01-15T19:09:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5580000000000001e-06 2021-01-15T19:09:08.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191871+0000", 2021-01-15T19:09:08.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0240000000000003e-06 2021-01-15T19:09:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191877+0000", 2021-01-15T19:09:08.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4040000000000002e-06 2021-01-15T19:09:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191879+0000", 2021-01-15T19:09:08.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046935880000000003 2021-01-15T19:09:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196573+0000", 2021-01-15T19:09:08.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000107329 2021-01-15T19:09:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.196680+0000", 2021-01-15T19:09:08.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047579799999999999 2021-01-15T19:09:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197156+0000", 2021-01-15T19:09:08.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003761206 2021-01-15T19:09:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.200917+0000", 2021-01-15T19:09:08.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044910649999999998 2021-01-15T19:09:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205408+0000", 2021-01-15T19:09:08.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050471800000000003 2021-01-15T19:09:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205913+0000", 2021-01-15T19:09:08.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014071523000000001 2021-01-15T19:09:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:366 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.263712071, 2021-01-15T19:09:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999, 2021-01-15T19:09:08.483 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.484 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 366 2021-01-15T19:09:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0789999999999999e-05 2021-01-15T19:09:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191920+0000", 2021-01-15T19:09:08.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2760000000000001e-06 2021-01-15T19:09:08.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191923+0000", 2021-01-15T19:09:08.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4840000000000001e-06 2021-01-15T19:09:08.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191930+0000", 2021-01-15T19:09:08.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2479999999999999e-06 2021-01-15T19:09:08.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191932+0000", 2021-01-15T19:09:08.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055448800000000003 2021-01-15T19:09:08.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197477+0000", 2021-01-15T19:09:08.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101512 2021-01-15T19:09:08.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:08.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197578+0000", 2021-01-15T19:09:08.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009253308 2021-01-15T19:09:08.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206832+0000", 2021-01-15T19:09:08.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011591699999999999 2021-01-15T19:09:08.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206947+0000", 2021-01-15T19:09:08.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012706729999999999 2021-01-15T19:09:08.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.208218+0000", 2021-01-15T19:09:08.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999 2021-01-15T19:09:08.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.496 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 50/21 ECSubWriteReply(tid=157, last_complete=50'18, committed=1, applied=1))", 2021-01-15T19:09:08.497 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:08.497 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.762682896000001, 2021-01-15T19:09:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001, 2021-01-15T19:09:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4277000000000002e-05 2021-01-15T19:09:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692952+0000", 2021-01-15T19:09:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015131444000000001 2021-01-15T19:09:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708084+0000", 2021-01-15T19:09:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9973999999999999e-05 2021-01-15T19:09:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708104+0000", 2021-01-15T19:09:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4741e-05 2021-01-15T19:09:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708119+0000", 2021-01-15T19:09:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2155000000000004e-05 2021-01-15T19:09:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708181+0000", 2021-01-15T19:09:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7632999999999999e-05 2021-01-15T19:09:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708198+0000", 2021-01-15T19:09:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001 2021-01-15T19:09:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.191480780999999, 2021-01-15T19:09:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:09:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:09:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:09:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:09:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:09:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:09:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:09:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:09:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:09:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:09:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:09:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:09:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:09:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:09:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:09:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:09:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:09:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:09:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:09:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:09:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:09:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:09:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.191069154000001, 2021-01-15T19:09:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:634 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~663552] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.255540798, 2021-01-15T19:09:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952, 2021-01-15T19:09:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 634 2021-01-15T19:09:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081858500000000004 2021-01-15T19:09:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200889+0000", 2021-01-15T19:09:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2700000000000004e-06 2021-01-15T19:09:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200897+0000", 2021-01-15T19:09:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7638e-05 2021-01-15T19:09:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200915+0000", 2021-01-15T19:09:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0730000000000005e-06 2021-01-15T19:09:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200923+0000", 2021-01-15T19:09:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6312999999999999e-05 2021-01-15T19:09:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200999+0000", 2021-01-15T19:09:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018274900000000001 2021-01-15T19:09:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.201182+0000", 2021-01-15T19:09:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001367224 2021-01-15T19:09:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.202549+0000", 2021-01-15T19:09:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012523378 2021-01-15T19:09:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215073+0000", 2021-01-15T19:09:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6226999999999999e-05 2021-01-15T19:09:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215169+0000", 2021-01-15T19:09:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131495 2021-01-15T19:09:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215300+0000", 2021-01-15T19:09:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952 2021-01-15T19:09:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:639 3.8s0 3:1009af39:::smithi00140323-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.247839668999999, 2021-01-15T19:09:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074, 2021-01-15T19:09:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 639 2021-01-15T19:09:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1959000000000003e-05 2021-01-15T19:09:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207813+0000", 2021-01-15T19:09:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7560000000000002e-06 2021-01-15T19:09:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207820+0000", 2021-01-15T19:09:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207832+0000", 2021-01-15T19:09:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.583e-06 2021-01-15T19:09:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207837+0000", 2021-01-15T19:09:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010014853000000001 2021-01-15T19:09:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217852+0000", 2021-01-15T19:09:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0618000000000006e-05 2021-01-15T19:09:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217942+0000", 2021-01-15T19:09:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035136210000000002 2021-01-15T19:09:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221456+0000", 2021-01-15T19:09:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116369 2021-01-15T19:09:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221572+0000", 2021-01-15T19:09:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001478222 2021-01-15T19:09:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.223051+0000", 2021-01-15T19:09:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074 2021-01-15T19:09:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:693 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.274483679999999, 2021-01-15T19:09:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668, 2021-01-15T19:09:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 693 2021-01-15T19:09:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2021-01-15T19:09:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181158+0000", 2021-01-15T19:09:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5740000000000001e-06 2021-01-15T19:09:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181165+0000", 2021-01-15T19:09:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2294e-05 2021-01-15T19:09:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181177+0000", 2021-01-15T19:09:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6979999999999998e-06 2021-01-15T19:09:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181185+0000", 2021-01-15T19:09:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.101e-05 2021-01-15T19:09:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181246+0000", 2021-01-15T19:09:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2619999999999999e-05 2021-01-15T19:09:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181258+0000", 2021-01-15T19:09:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075312950000000004 2021-01-15T19:09:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188790+0000", 2021-01-15T19:09:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1936999999999998e-05 2021-01-15T19:09:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188872+0000", 2021-01-15T19:09:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013017509999999999 2021-01-15T19:09:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.201889+0000", 2021-01-15T19:09:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668 2021-01-15T19:09:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.274396652, 2021-01-15T19:09:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.274332852000001, 2021-01-15T19:09:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:711 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~417792] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:09:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:59.238953+0000", 2021-01-15T19:09:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.216657734, 2021-01-15T19:09:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014839906999999999, 2021-01-15T19:09:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 711 2021-01-15T19:09:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.238953+0000", 2021-01-15T19:09:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.238953+0000", 2021-01-15T19:09:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00061497199999999998 2021-01-15T19:09:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239568+0000", 2021-01-15T19:09:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2340000000000002e-06 2021-01-15T19:09:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239574+0000", 2021-01-15T19:09:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1090000000000001e-05 2021-01-15T19:09:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239585+0000", 2021-01-15T19:09:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7209999999999999e-06 2021-01-15T19:09:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239590+0000", 2021-01-15T19:09:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3004999999999996e-05 2021-01-15T19:09:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239663+0000", 2021-01-15T19:09:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123344 2021-01-15T19:09:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.239787+0000", 2021-01-15T19:09:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099521799999999997 2021-01-15T19:09:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.240782+0000", 2021-01-15T19:09:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012820557999999999 2021-01-15T19:09:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253603+0000", 2021-01-15T19:09:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9769000000000001e-05 2021-01-15T19:09:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253682+0000", 2021-01-15T19:09:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011099600000000001 2021-01-15T19:09:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:59.253793+0000", 2021-01-15T19:09:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014839906999999999 2021-01-15T19:09:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:824 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27239280900000001, 2021-01-15T19:09:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001, 2021-01-15T19:09:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 824 2021-01-15T19:09:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6789000000000003e-05 2021-01-15T19:09:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183255+0000", 2021-01-15T19:09:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0429999999999996e-06 2021-01-15T19:09:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183264+0000", 2021-01-15T19:09:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7039000000000001e-05 2021-01-15T19:09:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183281+0000", 2021-01-15T19:09:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0779999999999998e-06 2021-01-15T19:09:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183288+0000", 2021-01-15T19:09:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056417399999999997 2021-01-15T19:09:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183852+0000", 2021-01-15T19:09:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086462300000000002 2021-01-15T19:09:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184717+0000", 2021-01-15T19:09:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019389661999999998 2021-01-15T19:09:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.204107+0000", 2021-01-15T19:09:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001 2021-01-15T19:09:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.271750086, 2021-01-15T19:09:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27158714499999997, 2021-01-15T19:09:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:830 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.271466767, 2021-01-15T19:09:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018941972000000001, 2021-01-15T19:09:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 830 2021-01-15T19:09:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7934e-05 2021-01-15T19:09:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184182+0000", 2021-01-15T19:09:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8520000000000004e-06 2021-01-15T19:09:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184190+0000", 2021-01-15T19:09:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2923e-05 2021-01-15T19:09:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184203+0000", 2021-01-15T19:09:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1710000000000001e-06 2021-01-15T19:09:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184208+0000", 2021-01-15T19:09:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010232240000000001 2021-01-15T19:09:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185231+0000", 2021-01-15T19:09:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013339 2021-01-15T19:09:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185365+0000", 2021-01-15T19:09:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087249099999999998 2021-01-15T19:09:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186237+0000", 2021-01-15T19:09:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014278872 2021-01-15T19:09:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200516+0000", 2021-01-15T19:09:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024695059999999998 2021-01-15T19:09:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.202986+0000", 2021-01-15T19:09:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100609 2021-01-15T19:09:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203086+0000", 2021-01-15T19:09:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018941972000000001 2021-01-15T19:09:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:831 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 19] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27135681, 2021-01-15T19:09:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019059996999999999, 2021-01-15T19:09:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 831 2021-01-15T19:09:08.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3042e-05 2021-01-15T19:09:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184287+0000", 2021-01-15T19:09:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8390000000000003e-06 2021-01-15T19:09:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184293+0000", 2021-01-15T19:09:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0827e-05 2021-01-15T19:09:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184304+0000", 2021-01-15T19:09:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6419999999999998e-06 2021-01-15T19:09:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184309+0000", 2021-01-15T19:09:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040504060000000003 2021-01-15T19:09:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188359+0000", 2021-01-15T19:09:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122912 2021-01-15T19:09:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188482+0000", 2021-01-15T19:09:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066735099999999995 2021-01-15T19:09:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189149+0000", 2021-01-15T19:09:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01136844 2021-01-15T19:09:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200518+0000", 2021-01-15T19:09:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027085680000000002 2021-01-15T19:09:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203226+0000", 2021-01-15T19:09:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.797e-05 2021-01-15T19:09:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203314+0000", 2021-01-15T19:09:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019059996999999999 2021-01-15T19:09:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:832 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.27126303200000002, 2021-01-15T19:09:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596, 2021-01-15T19:09:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 832 2021-01-15T19:09:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6316000000000003e-05 2021-01-15T19:09:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184384+0000", 2021-01-15T19:09:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:09:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184390+0000", 2021-01-15T19:09:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.133e-05 2021-01-15T19:09:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184401+0000", 2021-01-15T19:09:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7360000000000001e-06 2021-01-15T19:09:08.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184406+0000", 2021-01-15T19:09:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054429209999999999 2021-01-15T19:09:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189849+0000", 2021-01-15T19:09:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013384599999999999 2021-01-15T19:09:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189983+0000", 2021-01-15T19:09:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053970699999999995 2021-01-15T19:09:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.190523+0000", 2021-01-15T19:09:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099961230000000009 2021-01-15T19:09:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200519+0000", 2021-01-15T19:09:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029232770000000002 2021-01-15T19:09:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203442+0000", 2021-01-15T19:09:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8573999999999996e-05 2021-01-15T19:09:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203531+0000", 2021-01-15T19:09:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596 2021-01-15T19:09:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.7s0 80/52 ECSubRead(tid=294, to_read={3:e0e4bbc9:::smithi00140323-39:head=0,2355200,0}, subchunks={3:e0e4bbc9:::smithi00140323-39:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.26994886000000001, 2021-01-15T19:09:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016692684999999999, 2021-01-15T19:09:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:08.722 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6965000000000001e-05 2021-01-15T19:09:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185679+0000", 2021-01-15T19:09:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4359999999999996e-06 2021-01-15T19:09:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185687+0000", 2021-01-15T19:09:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011445029000000001 2021-01-15T19:09:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.197132+0000", 2021-01-15T19:09:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052232550000000004 2021-01-15T19:09:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.202355+0000", 2021-01-15T19:09:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016692684999999999 2021-01-15T19:09:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.7s0 80/52 ECSubRead(tid=296, to_read={3:e0e4bbc9:::smithi00140323-39:head=0,2355200,0}, subchunks={3:e0e4bbc9:::smithi00140323-39:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.186112+0000", 2021-01-15T19:09:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.26949925699999999, 2021-01-15T19:09:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01415631, 2021-01-15T19:09:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186112+0000", 2021-01-15T19:09:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186112+0000", 2021-01-15T19:09:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186112+0000", 2021-01-15T19:09:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186112+0000", 2021-01-15T19:09:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022227290000000001 2021-01-15T19:09:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188335+0000", 2021-01-15T19:09:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8173999999999999e-05 2021-01-15T19:09:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188353+0000", 2021-01-15T19:09:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010547793999999999 2021-01-15T19:09:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.198901+0000", 2021-01-15T19:09:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013676129999999999 2021-01-15T19:09:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200268+0000", 2021-01-15T19:09:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01415631 2021-01-15T19:09:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.7s0 80/52 ECSubReadReply(tid=294, attrs_read=0))", 2021-01-15T19:09:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.194043+0000", 2021-01-15T19:09:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.261567722, 2021-01-15T19:09:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015698659, 2021-01-15T19:09:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.194043+0000", 2021-01-15T19:09:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.194043+0000", 2021-01-15T19:09:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069922880000000001 2021-01-15T19:09:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201036+0000", 2021-01-15T19:09:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4779999999999997e-06 2021-01-15T19:09:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201045+0000", 2021-01-15T19:09:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8879999999999997e-06 2021-01-15T19:09:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201053+0000", 2021-01-15T19:09:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1740000000000002e-06 2021-01-15T19:09:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201057+0000", 2021-01-15T19:09:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046652129999999997 2021-01-15T19:09:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.205722+0000", 2021-01-15T19:09:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004019618 2021-01-15T19:09:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209742+0000", 2021-01-15T19:09:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015698659 2021-01-15T19:09:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:08.755 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:08.757 INFO:tasks.rados.rados.0.smithi001.stdout:302: done (6 left) 2021-01-15T19:09:08.757 INFO:tasks.rados.rados.0.smithi001.stdout:304: done (5 left) 2021-01-15T19:09:08.757 INFO:tasks.rados.rados.0.smithi001.stdout:305: done (4 left) 2021-01-15T19:09:08.758 INFO:tasks.rados.rados.0.smithi001.stdout:307: done (3 left) 2021-01-15T19:09:08.758 INFO:tasks.rados.rados.0.smithi001.stdout:308: done (2 left) 2021-01-15T19:09:08.758 INFO:tasks.rados.rados.0.smithi001.stdout:309: done (1 left) 2021-01-15T19:09:08.758 INFO:tasks.rados.rados.0.smithi001.stdout:310: done (0 left) 2021-01-15T19:09:08.758 INFO:tasks.rados.rados.0.smithi001.stdout:311: snap_create 2021-01-15T19:09:08.759 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:08.756+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:08.759 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:08.858 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:08.857+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:08.922 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:08.922 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:08.923 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:08.923 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:08.932 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:08.959 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:08.958+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:09.056 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:09.056 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:09.056 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:09.057 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:09.057 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:09.059 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:09.058+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:09.066 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:09.160 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:09.159+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:09.237 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:09.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.947313483999999, 2021-01-15T19:09:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:09.239 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.240 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:09.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:09.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:09.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:09.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:09.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:09.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:09.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:09.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:09.264 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.015550718, 2021-01-15T19:09:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:09.266 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.267 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:09.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:09.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:09.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.013262677, 2021-01-15T19:09:09.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:09.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:09.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:09.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:09.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.010673052000001, 2021-01-15T19:09:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:09.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:09.315 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.009764773999997, 2021-01-15T19:09:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.316 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.317 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:09.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:09.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:09.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:09.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.041080866999998, 2021-01-15T19:09:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:09.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.332 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:09.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:09.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:09.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:09.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:09.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:09.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:09.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:09.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:09.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:09.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:09.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:09.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:09.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:09.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:09.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:09.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:09.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:09.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:09.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:09.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:09.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:09.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:09.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:09.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:09.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:09.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:09.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:09.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.348 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:09.348 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:09.349 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.040931129999997, 2021-01-15T19:09:09.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:09.349 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.349 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.350 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:09.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.350 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.040779231999998, 2021-01-15T19:09:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.040653919999997, 2021-01-15T19:09:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.019821344999997, 2021-01-15T19:09:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.013528012999998, 2021-01-15T19:09:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.051468499999999, 2021-01-15T19:09:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.98909652, 2021-01-15T19:09:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.985794019, 2021-01-15T19:09:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:09.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.984472152999999, 2021-01-15T19:09:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.984101899999999, 2021-01-15T19:09:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.983767507, 2021-01-15T19:09:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.042506247999999, 2021-01-15T19:09:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0501433760000003, 2021-01-15T19:09:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.020149172, 2021-01-15T19:09:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:09.584 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:09.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:09.585 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:09:09.587 INFO:tasks.rados.rados.0.smithi001.stdout:311: done (0 left) 2021-01-15T19:09:09.587 INFO:tasks.rados.rados.0.smithi001.stdout:312: snap_remove snap 30 2021-01-15T19:09:09.587 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:09.584+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:09.588 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:09:09.686 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:09.684+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:09.751 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:09.753 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:09.753 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:09.754 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:09.754 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:09.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:09:09.774 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:09.786 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:09.784+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:09.793 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:09.814 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:09.835 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:09.856 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:09.876 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:09.888 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:09.885+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:09.889 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:09.889 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:09.898 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:09.899 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:09:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:09.988 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:09.987+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:10.027 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.027 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.036 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:10.089 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:10.089+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:10.162 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.162 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:10.218 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:10.217+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:10.291 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:10.290+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:10.298 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.298 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.307 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:10.327 INFO:tasks.rados.rados.0.smithi001.stdout:312: done (0 left) 2021-01-15T19:09:10.327 INFO:tasks.rados.rados.0.smithi001.stdout:313: rmattr oid 25 current snap is 33 2021-01-15T19:09:10.327 INFO:tasks.rados.rados.0.smithi001.stdout:313: done (0 left) 2021-01-15T19:09:10.328 INFO:tasks.rados.rados.0.smithi001.stdout:314: copy_from oid 15 from oid 38 current snap is 33 2021-01-15T19:09:10.328 INFO:tasks.rados.rados.0.smithi001.stdout:315: rollback oid 45 current snap is 33 2021-01-15T19:09:10.328 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 45 to 19 2021-01-15T19:09:10.328 INFO:tasks.rados.rados.0.smithi001.stdout:316: rmattr oid 1 current snap is 33 2021-01-15T19:09:10.329 INFO:tasks.rados.rados.0.smithi001.stdout:316: done (2 left) 2021-01-15T19:09:10.329 INFO:tasks.rados.rados.0.smithi001.stdout:317: rollback oid 42 current snap is 33 2021-01-15T19:09:10.329 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 42 to 16 2021-01-15T19:09:10.329 INFO:tasks.rados.rados.0.smithi001.stdout:318: snap_create 2021-01-15T19:09:10.331 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:09:10.331 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:10.331 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.19864689048395165 2021-01-15T19:09:10.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.19864689048395165 2021-01-15T19:09:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:09:10.342 INFO:tasks.rados.rados.0.smithi001.stdout:315: finishing rollback tid 1 to smithi00140323-45 2021-01-15T19:09:10.342 INFO:tasks.rados.rados.0.smithi001.stdout:315: finishing rollback tid 2 to smithi00140323-45 2021-01-15T19:09:10.342 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 120 (ObjNum 89 snap 16 seq_num 89) dirty exists 2021-01-15T19:09:10.343 INFO:tasks.rados.rados.0.smithi001.stdout:317: finishing rollback tid 0 to smithi00140323-42 2021-01-15T19:09:10.343 INFO:tasks.rados.rados.0.smithi001.stdout:317: finishing rollback tid 1 to smithi00140323-42 2021-01-15T19:09:10.347 INFO:tasks.rados.rados.0.smithi001.stdout:317: finishing rollback tid 2 to smithi00140323-42 2021-01-15T19:09:10.347 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 38 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:09:10.362 INFO:tasks.rados.rados.0.smithi001.stdout:314: finishing copy_from to smithi00140323-15 2021-01-15T19:09:10.362 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 15 v 53 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:10.363 INFO:tasks.rados.rados.0.smithi001.stdout:314: finishing copy_from racing read to smithi00140323-15 2021-01-15T19:09:10.392 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:10.390+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:10.473 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.473 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:10.527 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:10.526+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:10.594 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:10.593+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:10.617 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.617 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:10.696 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:10.695+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:10.797 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:10.796+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:10.800 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.800 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:10.900 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:10.898+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:10.954 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:10.954 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:10.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:11.000 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:10.999+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:11.088 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:11.088 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:11.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:11.140 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:11.139+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:11.202 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:11.201+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:11.228 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:11.228 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:11.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:11.310 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:11.306+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:11.328 INFO:tasks.rados.rados.0.smithi001.stdout:314: done (3 left) 2021-01-15T19:09:11.328 INFO:tasks.rados.rados.0.smithi001.stdout:315: done (2 left) 2021-01-15T19:09:11.329 INFO:tasks.rados.rados.0.smithi001.stdout:317: done (1 left) 2021-01-15T19:09:11.329 INFO:tasks.rados.rados.0.smithi001.stdout:318: done (0 left) 2021-01-15T19:09:11.329 INFO:tasks.rados.rados.0.smithi001.stdout:319: read oid 37 snap -1 2021-01-15T19:09:11.329 INFO:tasks.rados.rados.0.smithi001.stdout:319: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:11.330 INFO:tasks.rados.rados.0.smithi001.stdout:320: copy_from oid 39 from oid 4 current snap is 34 2021-01-15T19:09:11.330 INFO:tasks.rados.rados.0.smithi001.stdout:321: copy_from oid 7 from oid 34 current snap is 34 2021-01-15T19:09:11.330 INFO:tasks.rados.rados.0.smithi001.stdout:322: rollback oid 1 current snap is 34 2021-01-15T19:09:11.330 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 1 to 25 2021-01-15T19:09:11.331 INFO:tasks.rados.rados.0.smithi001.stdout:323: read oid 21 snap -1 2021-01-15T19:09:11.331 INFO:tasks.rados.rados.0.smithi001.stdout:323: expect (ObjNum 130 snap 31 seq_num 130) 2021-01-15T19:09:11.331 INFO:tasks.rados.rados.0.smithi001.stdout:324: rollback oid 49 current snap is 34 2021-01-15T19:09:11.331 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 49 to 22 2021-01-15T19:09:11.331 INFO:tasks.rados.rados.0.smithi001.stdout:325: snap_remove snap 16 2021-01-15T19:09:11.347 INFO:teuthology.orchestra.run.smithi001.stderr:reweighted osd.3 to 0.198647 (32da) 2021-01-15T19:09:11.408 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:11.407+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:11.426 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:11.427 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:11.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:11.509 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:11.508+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:11.573 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:11.574 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:11.588 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:11.611 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:11.609+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:11.717 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:11.710+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:11.756 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:11.757 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:11.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:11.812 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:11.811+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:11.918 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:11.920 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:11.919+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:11.944 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:11.945 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:11.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:12.039 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:12.038+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:12.069 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:12.090 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:12.109 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:12.123 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:12.120+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:12.130 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:12.132 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:12.133 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:12.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:12.163 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:12.173 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:12.199 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:12.219 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:12.220 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.220 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.221 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:12.234 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:12.233+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:12.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:12.340 INFO:tasks.rados.rados.0.smithi001.stdout:325: done (6 left) 2021-01-15T19:09:12.340 INFO:tasks.rados.rados.0.smithi001.stdout:326: copy_from oid 5 from oid 20 current snap is 34 2021-01-15T19:09:12.340 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 33 current snap is 34 2021-01-15T19:09:12.340 INFO:tasks.rados.rados.0.smithi001.stdout:327: seq_num 138 ranges {2039808=720896,2760704=540672,3301376=557056,3858432=507904} 2021-01-15T19:09:12.341 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:12.338+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:12.362 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:09:12.364 INFO:tasks.rados.rados.0.smithi001.stdout:327: writing smithi00140323-33 from 2039808 to 2760704 tid 1 2021-01-15T19:09:12.367 INFO:tasks.rados.rados.0.smithi001.stdout:327: writing smithi00140323-33 from 2760704 to 3301376 tid 2 2021-01-15T19:09:12.370 INFO:tasks.rados.rados.0.smithi001.stdout:327: writing smithi00140323-33 from 3301376 to 3858432 tid 3 2021-01-15T19:09:12.373 INFO:tasks.rados.rados.0.smithi001.stdout:327: writing smithi00140323-33 from 3858432 to 4366336 tid 4 2021-01-15T19:09:12.373 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 9 current snap is 34 2021-01-15T19:09:12.374 INFO:tasks.rados.rados.0.smithi001.stdout:324: finishing rollback tid 0 to smithi00140323-49 2021-01-15T19:09:12.374 INFO:tasks.rados.rados.0.smithi001.stdout:328: seq_num 139 ranges {139264=737280,876544=425984,1302528=761856,2064384=417792} 2021-01-15T19:09:12.378 INFO:tasks.rados.rados.0.smithi001.stdout:328: writing smithi00140323-9 from 139264 to 876544 tid 1 2021-01-15T19:09:12.380 INFO:tasks.rados.rados.0.smithi001.stdout:328: writing smithi00140323-9 from 876544 to 1302528 tid 2 2021-01-15T19:09:12.386 INFO:tasks.rados.rados.0.smithi001.stdout:328: writing smithi00140323-9 from 1302528 to 2064384 tid 3 2021-01-15T19:09:12.388 INFO:tasks.rados.rados.0.smithi001.stdout:328: writing smithi00140323-9 from 2064384 to 2482176 tid 4 2021-01-15T19:09:12.388 INFO:tasks.rados.rados.0.smithi001.stdout:329: setattr oid 2 current snap is 34 2021-01-15T19:09:12.389 INFO:tasks.rados.rados.0.smithi001.stdout:324: finishing rollback tid 1 to smithi00140323-49 2021-01-15T19:09:12.389 INFO:tasks.rados.rados.0.smithi001.stdout:324: finishing rollback tid 2 to smithi00140323-49 2021-01-15T19:09:12.389 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 60 (ObjNum 107 snap 22 seq_num 107) dirty exists 2021-01-15T19:09:12.392 INFO:tasks.rados.rados.0.smithi001.stdout:322: finishing rollback tid 0 to smithi00140323-1 2021-01-15T19:09:12.392 INFO:tasks.rados.rados.0.smithi001.stdout:322: finishing rollback tid 1 to smithi00140323-1 2021-01-15T19:09:12.393 INFO:tasks.rados.rados.0.smithi001.stdout:322: finishing rollback tid 2 to smithi00140323-1 2021-01-15T19:09:12.393 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 51 (ObjNum 47 snap 0 seq_num 47) dirty exists 2021-01-15T19:09:12.393 INFO:tasks.rados.rados.0.smithi001.stdout:323: expect (ObjNum 130 snap 31 seq_num 130) 2021-01-15T19:09:12.437 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:12.436+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:12.471 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:12.471 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:12.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:12.472 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:12.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.472 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:12.472 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:12.472 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.219957233000002, 2021-01-15T19:09:12.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:12.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:12.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.474 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:12.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:12.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:12.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:12.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:12.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:12.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:12.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:12.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:12.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:12.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:12.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:12.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:12.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:12.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:12.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:12.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:12.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:12.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:12.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:12.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:12.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.509 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.510 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:12.511 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.219766288999999, 2021-01-15T19:09:12.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:12.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:12.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:12.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:12.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:12.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:12.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:12.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:12.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:12.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:12.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:12.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:12.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.280814192000001, 2021-01-15T19:09:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:12.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:12.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:12.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:12.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:12.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:12.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:12.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:12.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:12.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:12.596 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.597 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.278093730999998, 2021-01-15T19:09:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:12.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:12.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:12.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:12.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:12.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:12.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:12.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:12.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:12.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:12.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:12.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:12.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:12.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:12.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:12.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:12.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:12.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:12.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:12.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:12.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:12.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:12.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:12.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:12.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:12.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:12.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:12.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:12.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.613 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.614 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.614 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:12.614 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.277981151999999, 2021-01-15T19:09:12.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:12.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.615 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:12.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.616 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:12.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:12.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:12.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:12.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:12.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:12.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:12.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:12.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:12.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:12.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:12.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.277851248000001, 2021-01-15T19:09:12.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:12.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:12.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:12.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:12.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:12.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:12.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:12.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:12.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:12.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:12.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:12.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:12.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:12.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:12.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:12.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:12.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:12.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:12.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:12.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:12.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:12.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:12.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:12.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:12.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:12.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:12.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.661 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.661 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:12.661 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.254645978999999, 2021-01-15T19:09:12.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:12.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.662 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:12.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:12.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:12.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:12.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:12.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:12.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:12.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:12.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:12.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:12.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:12.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:12.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:12.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:12.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:12.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:12.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:12.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:12.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:12.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:12.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:12.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:12.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.676 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.677 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:12.677 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.254527781, 2021-01-15T19:09:12.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:12.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:12.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.679 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:12.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:12.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:12.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:12.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:12.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:12.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:12.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:12.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:12.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:12.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:12.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:12.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:12.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:12.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:12.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:12.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:12.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:12.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:12.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:12.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:12.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:12.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:12.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.697 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:12.699 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.254415090999998, 2021-01-15T19:09:12.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:12.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:12.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.700 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:12.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:12.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:12.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:12.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:12.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:12.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:12.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:12.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:12.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:12.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:12.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:12.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:12.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:12.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:12.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:12.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:12.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:12.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:12.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:12.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:12.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:12.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:12.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.714 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:12.715 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:12.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.248813341000002, 2021-01-15T19:09:12.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:12.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:12.716 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:12.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:12.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:12.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:12.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:12.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:12.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:12.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:12.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:12.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:12.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:12.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:12.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:12.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:12.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:12.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.724 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.725 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:12.725 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.240344617000002, 2021-01-15T19:09:12.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:12.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.726 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:12.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.727 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:12.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:12.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:12.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:12.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:12.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:12.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:12.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:12.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:12.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:12.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:12.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:12.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:12.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:12.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:12.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:12.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:12.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.235826033999999, 2021-01-15T19:09:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:12.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:12.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:12.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:12.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:12.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:12.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:12.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:12.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:12.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:12.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:12.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:12.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:12.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:12.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:12.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:12.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:12.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:12.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:12.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:12.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:12.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:12.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:12.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.755 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.755 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:12.755 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.231418937000001, 2021-01-15T19:09:12.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:12.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:12.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.757 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:12.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:12.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:12.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:12.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:12.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:12.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:12.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:12.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:12.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:12.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:12.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:12.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:12.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:12.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:12.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:12.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:12.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:12.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:12.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:12.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:12.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:12.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:12.770 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.770 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.770 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.771 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:12.771 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:12.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.230936810999999, 2021-01-15T19:09:12.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:12.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.772 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:12.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.773 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:12.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:12.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:12.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:12.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:12.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:12.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:12.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:12.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:12.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:12.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:12.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:12.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:12.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:12.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:12.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:12.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:12.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:12.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:12.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:12.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:12.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.230808265, 2021-01-15T19:09:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:12.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:12.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:12.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:12.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:12.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:12.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:12.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:12.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:12.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:12.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.811 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.812 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:12.813 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:12.813 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.222359125000001, 2021-01-15T19:09:12.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:12.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:12.814 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:12.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:12.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:12.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:12.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:12.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:12.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:12.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:12.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:12.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:12.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:12.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:12.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:12.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:12.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:12.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.825 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:12.825 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:12.825 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.263684187000001, 2021-01-15T19:09:12.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:12.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:12.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.827 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:12.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:12.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:12.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:12.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:12.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:12.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:12.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:12.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:12.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:12.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:12.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:12.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:12.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:12.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:12.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:12.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:12.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:12.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:12.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:12.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:12.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:12.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:12.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:12.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:12.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:12.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:12.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:12.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.845 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.846 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:12.846 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:12.848 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.139499505, 2021-01-15T19:09:12.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:12.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:12.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:12.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:12.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:12.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:12.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:12.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:12.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:12.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:12.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:12.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:12.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:12.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:12.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:12.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:12.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:12.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:12.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:12.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:12.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:12.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.870 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.871 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:12.871 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:12.871 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.2781382170000004, 2021-01-15T19:09:12.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:12.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:12.872 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:12.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:12.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:12.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:12.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:12.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:12.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:12.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:12.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:12.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:12.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:12.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:12.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:12.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:12.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:12.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:12.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:12.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:12.886 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.286497539, 2021-01-15T19:09:12.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:12.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:12.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:12.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:12.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:12.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:12.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:12.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.889 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:12.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:12.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:12.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:12.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:12.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:12.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:12.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:12.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:12.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:12.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:12.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:12.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:12.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:12.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:12.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:12.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:12.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:12.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:12.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:12.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:12.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:12.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:12.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:12.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:12.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:12.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:12.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:12.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:12.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:12.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:12.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:12.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:12.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:12.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:12.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:12.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:12.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:12.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:12.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:12.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:12.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:12.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:12.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:12.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:12.919 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:12.920 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:12.921 INFO:tasks.rados.rados.0.smithi001.stdout:322: done (9 left) 2021-01-15T19:09:12.921 INFO:tasks.rados.rados.0.smithi001.stdout:323: done (8 left) 2021-01-15T19:09:12.922 INFO:tasks.rados.rados.0.smithi001.stdout:324: done (7 left) 2021-01-15T19:09:12.922 INFO:tasks.rados.rados.0.smithi001.stdout:330: delete oid 40 current snap is 34 2021-01-15T19:09:12.923 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:12.923 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:12.920+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:13.022 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:13.020+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:13.073 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:13.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3.effab4bd (undecoded) ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:13.074 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.58260747499999999, 2021-01-15T19:09:13.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58262414600000001, 2021-01-15T19:09:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "delayed", 2021-01-15T19:09:13.082 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:13.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:13.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:13.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:13.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:13.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:13.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000283362 2021-01-15T19:09:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3.3265d72d (undecoded) ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:13.094 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.715781797, 2021-01-15T19:09:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71584268600000001, 2021-01-15T19:09:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "delayed", 2021-01-15T19:09:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.096 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:13.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:13.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:13.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:13.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:13.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:13.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:13.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:13.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150593 2021-01-15T19:09:13.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.117 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.117 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.118 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3.3265d72d (undecoded) ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:13.118 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:13.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.71567434100000005, 2021-01-15T19:09:13.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71577254899999998, 2021-01-15T19:09:13.119 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.119 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "delayed", 2021-01-15T19:09:13.119 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.120 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:13.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.120 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:13.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:13.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:13.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:13.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:13.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:13.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:13.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:13.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:13.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:13.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:13.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:13.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:13.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:13.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:13.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014852100000000001 2021-01-15T19:09:13.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.130 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.130 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.130 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.133 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2021-01-15T19:09:13.133 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 3 2021-01-15T19:09:13.137 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:13.138 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:13.142 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:13.141+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:13.245 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:13.242+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:13.274 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:13.275 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:13.275 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:13.275 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:13.275 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:13.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:13.345 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:09:13.351 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:13.344+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:13.361 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 63 (ObjNum 115 snap 25 seq_num 115) dirty exists 2021-01-15T19:09:13.365 INFO:tasks.rados.rados.0.smithi001.stdout:328: finishing write tid 1 to smithi00140323-9 2021-01-15T19:09:13.366 INFO:tasks.rados.rados.0.smithi001.stdout:328: finishing write tid 2 to smithi00140323-9 2021-01-15T19:09:13.367 INFO:tasks.rados.rados.0.smithi001.stdout:328: finishing write tid 3 to smithi00140323-9 2021-01-15T19:09:13.371 INFO:tasks.rados.rados.0.smithi001.stdout:326: finishing copy_from to smithi00140323-5 2021-01-15T19:09:13.372 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 8 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:09:13.374 INFO:tasks.rados.rados.0.smithi001.stdout:328: finishing write tid 4 to smithi00140323-9 2021-01-15T19:09:13.374 INFO:tasks.rados.rados.0.smithi001.stdout:326: finishing copy_from racing read to smithi00140323-5 2021-01-15T19:09:13.376 INFO:tasks.rados.rados.0.smithi001.stdout:328: finishing write tid 5 to smithi00140323-9 2021-01-15T19:09:13.381 INFO:tasks.rados.rados.0.smithi001.stdout:328: finishing write tid 6 to smithi00140323-9 2021-01-15T19:09:13.381 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 9 v 40 (ObjNum 139 snap 34 seq_num 139) dirty exists 2021-01-15T19:09:13.381 INFO:tasks.rados.rados.0.smithi001.stdout:328: left oid 9 (ObjNum 139 snap 34 seq_num 139) 2021-01-15T19:09:13.399 INFO:tasks.rados.rados.0.smithi001.stdout:319: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:13.446 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:13.445+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:13.513 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:13.514 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:13.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:13.514 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:13.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.515 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:366 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:13.515 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:13.515 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.319331783999999, 2021-01-15T19:09:13.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999, 2021-01-15T19:09:13.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.516 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:13.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.517 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 366 2021-01-15T19:09:13.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.517 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:13.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:13.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0789999999999999e-05 2021-01-15T19:09:13.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191920+0000", 2021-01-15T19:09:13.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2760000000000001e-06 2021-01-15T19:09:13.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191923+0000", 2021-01-15T19:09:13.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4840000000000001e-06 2021-01-15T19:09:13.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191930+0000", 2021-01-15T19:09:13.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2479999999999999e-06 2021-01-15T19:09:13.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191932+0000", 2021-01-15T19:09:13.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055448800000000003 2021-01-15T19:09:13.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197477+0000", 2021-01-15T19:09:13.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101512 2021-01-15T19:09:13.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:13.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197578+0000", 2021-01-15T19:09:13.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009253308 2021-01-15T19:09:13.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206832+0000", 2021-01-15T19:09:13.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011591699999999999 2021-01-15T19:09:13.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:13.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206947+0000", 2021-01-15T19:09:13.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012706729999999999 2021-01-15T19:09:13.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:13.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.208218+0000", 2021-01-15T19:09:13.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999 2021-01-15T19:09:13.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 50/21 ECSubWriteReply(tid=157, last_complete=50'18, committed=1, applied=1))", 2021-01-15T19:09:13.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:13.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.818302609, 2021-01-15T19:09:13.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001, 2021-01-15T19:09:13.536 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:13.536 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:13.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692928+0000", 2021-01-15T19:09:13.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4277000000000002e-05 2021-01-15T19:09:13.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.692952+0000", 2021-01-15T19:09:13.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015131444000000001 2021-01-15T19:09:13.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708084+0000", 2021-01-15T19:09:13.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9973999999999999e-05 2021-01-15T19:09:13.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708104+0000", 2021-01-15T19:09:13.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4741e-05 2021-01-15T19:09:13.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708119+0000", 2021-01-15T19:09:13.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2155000000000004e-05 2021-01-15T19:09:13.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708181+0000", 2021-01-15T19:09:13.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7632999999999999e-05 2021-01-15T19:09:13.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:13.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:38.708198+0000", 2021-01-15T19:09:13.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015270224000000001 2021-01-15T19:09:13.548 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.549 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.549 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.550 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:13.550 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:13.550 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.247100494000001, 2021-01-15T19:09:13.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:09:13.550 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:13.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.947 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:09:13.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:13.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:13.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:09:13.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:09:13.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:13.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:09:13.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:09:13.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:09:13.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:09:13.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:09:13.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:09:13.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:09:13.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:09:13.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:13.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:09:13.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:09:13.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:13.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:09:13.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:09:13.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:13.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:09:13.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:09:13.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:13.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:09:13.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:09:13.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:13.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:09:13.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:09:13.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.968 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:13.969 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:13.969 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.246688867000003, 2021-01-15T19:09:13.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:13.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:13.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:13.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:13.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:13.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:13.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:13.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:13.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:13.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:13.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:13.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:13.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:13.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:13.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:13.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:13.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:13.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:13.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:13.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:13.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:13.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:13.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:13.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:13.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:13.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:13.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:13.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:13.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:13.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:13.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:13.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:13.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.986 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:13.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:13.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:634 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~663552] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:13.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:13.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.311160511000001, 2021-01-15T19:09:13.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952, 2021-01-15T19:09:13.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:13.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:13.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:13.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:13.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:13.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 634 2021-01-15T19:09:13.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:13.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:13.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:13.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:13.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:13.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200070+0000", 2021-01-15T19:09:13.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081858500000000004 2021-01-15T19:09:13.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:13.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200889+0000", 2021-01-15T19:09:13.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2700000000000004e-06 2021-01-15T19:09:13.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:13.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200897+0000", 2021-01-15T19:09:13.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7638e-05 2021-01-15T19:09:13.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:13.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200915+0000", 2021-01-15T19:09:13.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0730000000000005e-06 2021-01-15T19:09:13.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:13.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200923+0000", 2021-01-15T19:09:13.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6312999999999999e-05 2021-01-15T19:09:13.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:13.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.200999+0000", 2021-01-15T19:09:13.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018274900000000001 2021-01-15T19:09:13.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:13.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.201182+0000", 2021-01-15T19:09:13.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001367224 2021-01-15T19:09:13.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:13.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.202549+0000", 2021-01-15T19:09:13.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012523378 2021-01-15T19:09:13.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:13.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:13.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:13.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215073+0000", 2021-01-15T19:09:14.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6226999999999999e-05 2021-01-15T19:09:14.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:14.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215169+0000", 2021-01-15T19:09:14.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131495 2021-01-15T19:09:14.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.215300+0000", 2021-01-15T19:09:14.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229952 2021-01-15T19:09:14.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.002 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.003 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:639 3.8s0 3:1009af39:::smithi00140323-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:14.003 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:14.003 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.303459382, 2021-01-15T19:09:14.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074, 2021-01-15T19:09:14.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.005 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 639 2021-01-15T19:09:14.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:14.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207771+0000", 2021-01-15T19:09:14.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1959000000000003e-05 2021-01-15T19:09:14.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207813+0000", 2021-01-15T19:09:14.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7560000000000002e-06 2021-01-15T19:09:14.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207820+0000", 2021-01-15T19:09:14.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:14.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207832+0000", 2021-01-15T19:09:14.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.583e-06 2021-01-15T19:09:14.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.207837+0000", 2021-01-15T19:09:14.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010014853000000001 2021-01-15T19:09:14.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217852+0000", 2021-01-15T19:09:14.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0618000000000006e-05 2021-01-15T19:09:14.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:14.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.217942+0000", 2021-01-15T19:09:14.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035136210000000002 2021-01-15T19:09:14.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221456+0000", 2021-01-15T19:09:14.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116369 2021-01-15T19:09:14.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.221572+0000", 2021-01-15T19:09:14.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001478222 2021-01-15T19:09:14.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.223051+0000", 2021-01-15T19:09:14.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015279074 2021-01-15T19:09:14.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.025 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.026 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:693 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:14.026 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:14.026 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.330103393, 2021-01-15T19:09:14.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668, 2021-01-15T19:09:14.026 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.027 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.027 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.027 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.027 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.027 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 693 2021-01-15T19:09:14.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.028 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:14.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:14.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2021-01-15T19:09:14.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181158+0000", 2021-01-15T19:09:14.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5740000000000001e-06 2021-01-15T19:09:14.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181165+0000", 2021-01-15T19:09:14.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2294e-05 2021-01-15T19:09:14.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181177+0000", 2021-01-15T19:09:14.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6979999999999998e-06 2021-01-15T19:09:14.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181185+0000", 2021-01-15T19:09:14.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.101e-05 2021-01-15T19:09:14.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181246+0000", 2021-01-15T19:09:14.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2619999999999999e-05 2021-01-15T19:09:14.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:14.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181258+0000", 2021-01-15T19:09:14.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075312950000000004 2021-01-15T19:09:14.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188790+0000", 2021-01-15T19:09:14.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1936999999999998e-05 2021-01-15T19:09:14.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188872+0000", 2021-01-15T19:09:14.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013017509999999999 2021-01-15T19:09:14.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.201889+0000", 2021-01-15T19:09:14.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668 2021-01-15T19:09:14.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.042 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:14.043 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:14.043 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.330016365000001, 2021-01-15T19:09:14.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:14.043 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.044 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.045 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:14.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.048 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:14.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:14.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:14.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:14.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:14.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:14.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:14.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:14.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:14.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:14.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:14.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:14.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:14.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:14.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:14.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:14.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:14.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:14.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:14.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:14.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:14.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:14.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.059 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.060 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:14.060 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:14.061 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.329952564999999, 2021-01-15T19:09:14.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:14.061 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.061 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.061 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.062 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:14.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.062 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:14.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:14.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:14.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:14.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:14.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:14.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:14.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:14.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:14.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:14.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:14.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:14.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:14.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:14.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:14.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:14.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:14.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:14.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:14.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:14.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:14.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:14.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.075 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.076 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:824 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:14.076 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:14.076 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3280125219999999, 2021-01-15T19:09:14.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001, 2021-01-15T19:09:14.077 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.077 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.078 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 824 2021-01-15T19:09:14.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.079 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:14.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6789000000000003e-05 2021-01-15T19:09:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183255+0000", 2021-01-15T19:09:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0429999999999996e-06 2021-01-15T19:09:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183264+0000", 2021-01-15T19:09:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7039000000000001e-05 2021-01-15T19:09:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183281+0000", 2021-01-15T19:09:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0779999999999998e-06 2021-01-15T19:09:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183288+0000", 2021-01-15T19:09:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056417399999999997 2021-01-15T19:09:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183852+0000", 2021-01-15T19:09:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086462300000000002 2021-01-15T19:09:14.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184717+0000", 2021-01-15T19:09:14.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019389661999999998 2021-01-15T19:09:14.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.204107+0000", 2021-01-15T19:09:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001 2021-01-15T19:09:14.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.089 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3273697990000004, 2021-01-15T19:09:14.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:14.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:14.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:14.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:14.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:14.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:14.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:14.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:14.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3272068580000003, 2021-01-15T19:09:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:14.103 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.104 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:14.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:14.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:14.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:14.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:14.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:14.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:14.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:14.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.114 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.115 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:830 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:14.115 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:14.115 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3270864800000002, 2021-01-15T19:09:14.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018941972000000001, 2021-01-15T19:09:14.116 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.116 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:14.116 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.116 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.117 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.117 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 830 2021-01-15T19:09:14.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.117 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:14.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:14.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7934e-05 2021-01-15T19:09:14.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184182+0000", 2021-01-15T19:09:14.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8520000000000004e-06 2021-01-15T19:09:14.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184190+0000", 2021-01-15T19:09:14.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2923e-05 2021-01-15T19:09:14.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184203+0000", 2021-01-15T19:09:14.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1710000000000001e-06 2021-01-15T19:09:14.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184208+0000", 2021-01-15T19:09:14.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010232240000000001 2021-01-15T19:09:14.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185231+0000", 2021-01-15T19:09:14.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013339 2021-01-15T19:09:14.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185365+0000", 2021-01-15T19:09:14.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087249099999999998 2021-01-15T19:09:14.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:14.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186237+0000", 2021-01-15T19:09:14.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014278872 2021-01-15T19:09:14.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:14.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200516+0000", 2021-01-15T19:09:14.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024695059999999998 2021-01-15T19:09:14.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:14.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.202986+0000", 2021-01-15T19:09:14.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100609 2021-01-15T19:09:14.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203086+0000", 2021-01-15T19:09:14.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018941972000000001 2021-01-15T19:09:14.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.132 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:831 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 19] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:14.132 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:14.132 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3269765229999999, 2021-01-15T19:09:14.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019059996999999999, 2021-01-15T19:09:14.133 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.133 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:14.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.133 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.134 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.134 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 831 2021-01-15T19:09:14.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.134 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:14.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:14.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3042e-05 2021-01-15T19:09:14.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184287+0000", 2021-01-15T19:09:14.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8390000000000003e-06 2021-01-15T19:09:14.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184293+0000", 2021-01-15T19:09:14.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0827e-05 2021-01-15T19:09:14.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184304+0000", 2021-01-15T19:09:14.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6419999999999998e-06 2021-01-15T19:09:14.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184309+0000", 2021-01-15T19:09:14.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040504060000000003 2021-01-15T19:09:14.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188359+0000", 2021-01-15T19:09:14.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122912 2021-01-15T19:09:14.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188482+0000", 2021-01-15T19:09:14.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066735099999999995 2021-01-15T19:09:14.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:14.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189149+0000", 2021-01-15T19:09:14.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01136844 2021-01-15T19:09:14.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:14.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200518+0000", 2021-01-15T19:09:14.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027085680000000002 2021-01-15T19:09:14.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:14.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203226+0000", 2021-01-15T19:09:14.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.797e-05 2021-01-15T19:09:14.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203314+0000", 2021-01-15T19:09:14.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019059996999999999 2021-01-15T19:09:14.146 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.147 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.147 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:832 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:14.148 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:14.148 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3268827449999998, 2021-01-15T19:09:14.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596, 2021-01-15T19:09:14.148 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.148 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:14.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.149 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.149 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 832 2021-01-15T19:09:14.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.150 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:14.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:14.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6316000000000003e-05 2021-01-15T19:09:14.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184384+0000", 2021-01-15T19:09:14.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:09:14.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184390+0000", 2021-01-15T19:09:14.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.133e-05 2021-01-15T19:09:14.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184401+0000", 2021-01-15T19:09:14.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7360000000000001e-06 2021-01-15T19:09:14.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184406+0000", 2021-01-15T19:09:14.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054429209999999999 2021-01-15T19:09:14.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189849+0000", 2021-01-15T19:09:14.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013384599999999999 2021-01-15T19:09:14.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189983+0000", 2021-01-15T19:09:14.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053970699999999995 2021-01-15T19:09:14.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:14.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.190523+0000", 2021-01-15T19:09:14.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099961230000000009 2021-01-15T19:09:14.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:14.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200519+0000", 2021-01-15T19:09:14.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029232770000000002 2021-01-15T19:09:14.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:14.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203442+0000", 2021-01-15T19:09:14.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8573999999999996e-05 2021-01-15T19:09:14.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203531+0000", 2021-01-15T19:09:14.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596 2021-01-15T19:09:14.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.174 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.174 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.175 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.7s0 80/52 ECSubRead(tid=294, to_read={3:e0e4bbc9:::smithi00140323-39:head=0,2355200,0}, subchunks={3:e0e4bbc9:::smithi00140323-39:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:14.175 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:14.175 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.325568573, 2021-01-15T19:09:14.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016692684999999999, 2021-01-15T19:09:14.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:14.177 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:14.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:14.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:14.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:14.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6965000000000001e-05 2021-01-15T19:09:14.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185679+0000", 2021-01-15T19:09:14.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4359999999999996e-06 2021-01-15T19:09:14.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185687+0000", 2021-01-15T19:09:14.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011445029000000001 2021-01-15T19:09:14.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.197132+0000", 2021-01-15T19:09:14.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052232550000000004 2021-01-15T19:09:14.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.202355+0000", 2021-01-15T19:09:14.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016692684999999999 2021-01-15T19:09:14.212 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.212 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.214 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.7s0 80/52 ECSubReadReply(tid=294, attrs_read=0))", 2021-01-15T19:09:14.214 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.194043+0000", 2021-01-15T19:09:14.214 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3171874350000001, 2021-01-15T19:09:14.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015698659, 2021-01-15T19:09:14.215 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.215 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:14.216 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.194043+0000", 2021-01-15T19:09:14.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.194043+0000", 2021-01-15T19:09:14.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069922880000000001 2021-01-15T19:09:14.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201036+0000", 2021-01-15T19:09:14.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4779999999999997e-06 2021-01-15T19:09:14.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201045+0000", 2021-01-15T19:09:14.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8879999999999997e-06 2021-01-15T19:09:14.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201053+0000", 2021-01-15T19:09:14.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1740000000000002e-06 2021-01-15T19:09:14.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.201057+0000", 2021-01-15T19:09:14.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046652129999999997 2021-01-15T19:09:14.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.205722+0000", 2021-01-15T19:09:14.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004019618 2021-01-15T19:09:14.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209742+0000", 2021-01-15T19:09:14.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015698659 2021-01-15T19:09:14.237 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.238 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.238 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.239 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:14.239 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:14.239 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.166164212, 2021-01-15T19:09:14.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:14.240 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:14.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.242 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:14.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:14.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:14.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:14.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:14.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:14.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:14.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:14.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:14.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:14.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:14.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:14.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:14.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:14.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:14.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:14.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:14.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:14.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:14.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:14.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:14.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:14.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:14.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:14.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:14.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:14.287 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:14.287 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1660567559999999, 2021-01-15T19:09:14.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:14.288 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.288 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:14.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.289 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:14.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.290 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:14.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:14.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:14.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:14.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:14.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:14.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:14.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:14.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:14.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:14.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:14.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:14.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:14.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:14.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:14.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:14.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:14.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:14.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:14.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:14.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:14.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:14.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:14.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:14.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:14.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:14.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:14.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:14.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:14.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:14.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:14.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:14.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.315 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:14.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:14.318 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.0329898900000001, 2021-01-15T19:09:14.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:14.319 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.320 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:14.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.322 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:14.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.322 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:14.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:14.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:14.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:14.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:14.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:14.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:14.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:14.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.366 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:14.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:14.370 INFO:tasks.rados.rados.0.smithi001.stdout:320: finishing copy_from to smithi00140323-39 2021-01-15T19:09:14.371 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 45 (ObjNum 3 snap 0 seq_num 3) dirty exists 2021-01-15T19:09:14.371 INFO:tasks.rados.rados.0.smithi001.stdout:320: finishing copy_from racing read to smithi00140323-39 2021-01-15T19:09:14.371 INFO:tasks.rados.rados.0.smithi001.stdout:327: finishing write tid 1 to smithi00140323-33 2021-01-15T19:09:14.372 INFO:tasks.rados.rados.0.smithi001.stdout:327: finishing write tid 2 to smithi00140323-33 2021-01-15T19:09:14.372 INFO:tasks.rados.rados.0.smithi001.stdout:327: finishing write tid 3 to smithi00140323-33 2021-01-15T19:09:14.372 INFO:tasks.rados.rados.0.smithi001.stdout:321: finishing copy_from to smithi00140323-7 2021-01-15T19:09:14.372 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 110 (ObjNum 135 snap 32 seq_num 135) dirty exists 2021-01-15T19:09:14.373 INFO:tasks.rados.rados.0.smithi001.stdout:321: finishing copy_from racing read to smithi00140323-7 2021-01-15T19:09:14.373 INFO:tasks.rados.rados.0.smithi001.stdout:327: finishing write tid 4 to smithi00140323-33 2021-01-15T19:09:14.373 INFO:tasks.rados.rados.0.smithi001.stdout:327: finishing write tid 5 to smithi00140323-33 2021-01-15T19:09:14.374 INFO:tasks.rados.rados.0.smithi001.stdout:327: finishing write tid 6 to smithi00140323-33 2021-01-15T19:09:14.375 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 132 (ObjNum 138 snap 34 seq_num 138) dirty exists 2021-01-15T19:09:14.376 INFO:tasks.rados.rados.0.smithi001.stdout:327: left oid 33 (ObjNum 138 snap 34 seq_num 138) 2021-01-15T19:09:14.376 INFO:tasks.rados.rados.0.smithi001.stdout:319: done (7 left) 2021-01-15T19:09:14.377 INFO:tasks.rados.rados.0.smithi001.stdout:320: done (6 left) 2021-01-15T19:09:14.377 INFO:tasks.rados.rados.0.smithi001.stdout:321: done (5 left) 2021-01-15T19:09:14.377 INFO:tasks.rados.rados.0.smithi001.stdout:326: done (4 left) 2021-01-15T19:09:14.378 INFO:tasks.rados.rados.0.smithi001.stdout:327: done (3 left) 2021-01-15T19:09:14.378 INFO:tasks.rados.rados.0.smithi001.stdout:328: done (2 left) 2021-01-15T19:09:14.378 INFO:tasks.rados.rados.0.smithi001.stdout:329: done (1 left) 2021-01-15T19:09:14.379 INFO:tasks.rados.rados.0.smithi001.stdout:330: done (0 left) 2021-01-15T19:09:14.379 INFO:tasks.rados.rados.0.smithi001.stdout:331: snap_remove snap 19 2021-01-15T19:09:14.379 INFO:tasks.rados.rados.0.smithi001.stdout:331: done (0 left) 2021-01-15T19:09:14.380 INFO:tasks.rados.rados.0.smithi001.stdout:332: delete oid 39 current snap is 34 2021-01-15T19:09:14.380 INFO:tasks.rados.rados.0.smithi001.stdout:332: done (0 left) 2021-01-15T19:09:14.381 INFO:tasks.rados.rados.0.smithi001.stdout:333: read oid 11 snap 25 2021-01-15T19:09:14.381 INFO:tasks.rados.rados.0.smithi001.stdout:333: expect (ObjNum 102 snap 16 seq_num 102) 2021-01-15T19:09:14.382 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 23 current snap is 34 2021-01-15T19:09:14.382 INFO:tasks.rados.rados.0.smithi001.stdout:334: seq_num 142 ranges {376832=786432,1163264=475136,1638400=540672,2179072=589824} 2021-01-15T19:09:14.383 INFO:tasks.rados.rados.0.smithi001.stdout:334: writing smithi00140323-23 from 376832 to 1163264 tid 1 2021-01-15T19:09:14.383 INFO:tasks.rados.rados.0.smithi001.stdout:334: writing smithi00140323-23 from 1163264 to 1638400 tid 2 2021-01-15T19:09:14.383 INFO:tasks.rados.rados.0.smithi001.stdout:334: writing smithi00140323-23 from 1638400 to 2179072 tid 3 2021-01-15T19:09:14.384 INFO:tasks.rados.rados.0.smithi001.stdout:334: writing smithi00140323-23 from 2179072 to 2768896 tid 4 2021-01-15T19:09:14.384 INFO:tasks.rados.rados.0.smithi001.stdout:335: setattr oid 48 current snap is 34 2021-01-15T19:09:14.386 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:14.367+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:14.387 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:14.395 INFO:tasks.rados.rados.0.smithi001.stdout:336: copy_from oid 31 from oid 36 current snap is 34 2021-01-15T19:09:14.396 INFO:tasks.rados.rados.0.smithi001.stdout:337: read oid 27 snap -1 2021-01-15T19:09:14.396 INFO:tasks.rados.rados.0.smithi001.stdout:337: expect (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:09:14.396 INFO:tasks.rados.rados.0.smithi001.stdout:338: setattr oid 15 current snap is 34 2021-01-15T19:09:14.398 INFO:tasks.rados.rados.0.smithi001.stdout:339: read oid 34 snap -1 2021-01-15T19:09:14.398 INFO:tasks.rados.rados.0.smithi001.stdout:339: expect (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:09:14.399 INFO:tasks.rados.rados.0.smithi001.stdout:340: read oid 38 snap 27 2021-01-15T19:09:14.399 INFO:tasks.rados.rados.0.smithi001.stdout:340: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:14.400 INFO:tasks.rados.rados.0.smithi001.stdout:341: rollback oid 24 current snap is 34 2021-01-15T19:09:14.403 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 24 to 14 2021-01-15T19:09:14.403 INFO:tasks.rados.rados.0.smithi001.stdout:342: read oid 47 snap -1 2021-01-15T19:09:14.404 INFO:tasks.rados.rados.0.smithi001.stdout:342: expect (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:09:14.404 INFO:tasks.rados.rados.0.smithi001.stdout:343: rollback oid 42 current snap is 34 2021-01-15T19:09:14.405 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 42 to 23 2021-01-15T19:09:14.406 INFO:tasks.rados.rados.0.smithi001.stdout:344: delete oid 46 current snap is 34 2021-01-15T19:09:14.407 INFO:tasks.rados.rados.0.smithi001.stdout:337: expect (ObjNum 37 snap 0 seq_num 37) 2021-01-15T19:09:14.411 INFO:tasks.rados.rados.0.smithi001.stdout:343: finishing rollback tid 0 to smithi00140323-42 2021-01-15T19:09:14.411 INFO:tasks.rados.rados.0.smithi001.stdout:343: finishing rollback tid 1 to smithi00140323-42 2021-01-15T19:09:14.412 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 100 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:14.412 INFO:tasks.rados.rados.0.smithi001.stdout:343: finishing rollback tid 2 to smithi00140323-42 2021-01-15T19:09:14.413 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 46 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:09:14.414 INFO:tasks.rados.rados.0.smithi001.stdout:340: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:14.472 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:14.469+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:14.543 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:14.544 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:14.552 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:14.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.553 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:910 3.18s0 3.e3171838 (undecoded) ondisk+write+peerstat_old+known_if_redirected e86)", 2021-01-15T19:09:14.553 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:14.373864+0000", 2021-01-15T19:09:14.553 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.168635337, 2021-01-15T19:09:14.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.16865756400000001, 2021-01-15T19:09:14.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:14.554 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "delayed", 2021-01-15T19:09:14.554 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:14.554 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:14.554 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:14.555 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 910 2021-01-15T19:09:14.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:14.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:14.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373864+0000", 2021-01-15T19:09:14.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:14.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:14.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373864+0000", 2021-01-15T19:09:14.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0148999999999998e-05 2021-01-15T19:09:14.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:14.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373904+0000", 2021-01-15T19:09:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9809999999999992e-06 2021-01-15T19:09:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:14.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373913+0000", 2021-01-15T19:09:14.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5204999999999999e-05 2021-01-15T19:09:14.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:14.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373929+0000", 2021-01-15T19:09:14.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6420000000000001e-06 2021-01-15T19:09:14.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:14.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373935+0000", 2021-01-15T19:09:14.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8341999999999998e-05 2021-01-15T19:09:14.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:14.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373994+0000", 2021-01-15T19:09:14.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9780000000000001e-06 2021-01-15T19:09:14.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:14.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:14.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:14.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.373997+0000", 2021-01-15T19:09:14.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132297 2021-01-15T19:09:14.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.566 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:14.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:14.567 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2021-01-15T19:09:14.567 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2021-01-15T19:09:14.567 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:14.568 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:14.569 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:14.581 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:14.601 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:14.610 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:14.609+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:14.615 INFO:tasks.rados.rados.0.smithi001.stdout:336: finishing copy_from to smithi00140323-31 2021-01-15T19:09:14.616 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:14.616 INFO:tasks.rados.rados.0.smithi001.stdout:336: finishing copy_from racing read to smithi00140323-31 2021-01-15T19:09:14.623 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:14.644 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:14.666 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:14.672 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:14.671+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:14.687 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:14.689 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.690 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.690 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.690 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.690 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.691 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.691 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.691 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:14.738 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:14.738 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:14.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:14.794 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:14.793+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:14.875 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:14.874+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:14.877 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:14.878 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:14.889 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:09:14.976 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:14.975+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:15.054 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.054 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.064 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:09:15.077 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:15.076+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:15.178 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:15.177+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.189 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.198 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:09:15.279 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:15.278+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:15.362 INFO:tasks.rados.rados.0.smithi001.stdout:341: finishing rollback tid 0 to smithi00140323-24 2021-01-15T19:09:15.365 INFO:tasks.rados.rados.0.smithi001.stdout:341: finishing rollback tid 1 to smithi00140323-24 2021-01-15T19:09:15.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:09:15.368 INFO:tasks.rados.rados.0.smithi001.stdout:341: finishing rollback tid 2 to smithi00140323-24 2021-01-15T19:09:15.368 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 24 v 50 (ObjNum 37 snap 0 seq_num 37) dirty exists 2021-01-15T19:09:15.373 INFO:tasks.rados.rados.0.smithi001.stdout:335: done (11 left) 2021-01-15T19:09:15.374 INFO:tasks.rados.rados.0.smithi001.stdout:336: done (10 left) 2021-01-15T19:09:15.374 INFO:tasks.rados.rados.0.smithi001.stdout:337: done (9 left) 2021-01-15T19:09:15.374 INFO:tasks.rados.rados.0.smithi001.stdout:340: done (8 left) 2021-01-15T19:09:15.374 INFO:tasks.rados.rados.0.smithi001.stdout:341: done (7 left) 2021-01-15T19:09:15.375 INFO:tasks.rados.rados.0.smithi001.stdout:343: done (6 left) 2021-01-15T19:09:15.375 INFO:tasks.rados.rados.0.smithi001.stdout:344: done (5 left) 2021-01-15T19:09:15.375 INFO:tasks.rados.rados.0.smithi001.stdout:345: rollback oid 4 current snap is 34 2021-01-15T19:09:15.375 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 4 to 22 2021-01-15T19:09:15.375 INFO:tasks.rados.rados.0.smithi001.stdout:346: rollback oid 14 current snap is 34 2021-01-15T19:09:15.376 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 14 to 23 2021-01-15T19:09:15.376 INFO:tasks.rados.rados.0.smithi001.stdout:347: snap_remove snap 31 2021-01-15T19:09:15.383 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:15.378+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:15.394 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.395 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:15.480 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:15.479+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:15.532 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.532 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.541 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:15.582 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:15.581+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:15.682 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:15.681+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:15.707 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.707 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:15.783 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:15.782+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:15.841 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.841 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:15.884 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:15.883+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:15.985 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:15.983+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:15.985 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:15.986 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:15.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:16.085 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:16.084+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:16.168 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:16.168 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:16.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:16.225 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:16.224+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:16.287 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:16.286+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:16.310 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:16.311 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:16.320 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:16.345 INFO:tasks.rados.rados.0.smithi001.stdout:347: done (7 left) 2021-01-15T19:09:16.345 INFO:tasks.rados.rados.0.smithi001.stdout:348: snap_create 2021-01-15T19:09:16.345 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 15 v 54 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:16.345 INFO:tasks.rados.rados.0.smithi001.stdout:334: finishing write tid 1 to smithi00140323-23 2021-01-15T19:09:16.346 INFO:tasks.rados.rados.0.smithi001.stdout:333: expect (ObjNum 102 snap 16 seq_num 102) 2021-01-15T19:09:16.348 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:09:16.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:09:16.366 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:16.366 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:09:16.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2021-01-15T19:09:16.388 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:16.387+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:16.414 INFO:tasks.rados.rados.0.smithi001.stdout:346: finishing rollback tid 1 to smithi00140323-14 2021-01-15T19:09:16.415 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 70 (ObjNum 0 snap 0 seq_num 0) dirty dne 2021-01-15T19:09:16.415 INFO:tasks.rados.rados.0.smithi001.stdout:334: finishing write tid 2 to smithi00140323-23 2021-01-15T19:09:16.415 INFO:tasks.rados.rados.0.smithi001.stdout:334: finishing write tid 3 to smithi00140323-23 2021-01-15T19:09:16.415 INFO:tasks.rados.rados.0.smithi001.stdout:334: finishing write tid 4 to smithi00140323-23 2021-01-15T19:09:16.416 INFO:tasks.rados.rados.0.smithi001.stdout:334: finishing write tid 5 to smithi00140323-23 2021-01-15T19:09:16.416 INFO:tasks.rados.rados.0.smithi001.stdout:334: finishing write tid 6 to smithi00140323-23 2021-01-15T19:09:16.416 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 23 v 73 (ObjNum 142 snap 34 seq_num 142) dirty exists 2021-01-15T19:09:16.416 INFO:tasks.rados.rados.0.smithi001.stdout:334: left oid 23 (ObjNum 142 snap 34 seq_num 142) 2021-01-15T19:09:16.417 INFO:tasks.rados.rados.0.smithi001.stdout:342: expect (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:09:16.489 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:16.488+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:16.496 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:16.496 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:16.505 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:16.542 INFO:tasks.rados.rados.0.smithi001.stdout:339: expect (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:09:16.591 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:16.590+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:16.676 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:16.677 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:16.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:16.694 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:16.692+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:16.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:16.745 INFO:tasks.ceph.mon.a.smithi001.stderr:2021-01-15T19:09:16.744+0000 7f9995812700 -1 mon.a@0(leader).osd e88 definitely_dead 0 2021-01-15T19:09:16.751 INFO:tasks.rados.rados.0.smithi001.stdout:345: finishing rollback tid 0 to smithi00140323-4 2021-01-15T19:09:16.751 INFO:tasks.rados.rados.0.smithi001.stdout:345: finishing rollback tid 1 to smithi00140323-4 2021-01-15T19:09:16.751 INFO:tasks.rados.rados.0.smithi001.stdout:345: finishing rollback tid 2 to smithi00140323-4 2021-01-15T19:09:16.752 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 59 (ObjNum 3 snap 0 seq_num 3) dirty exists 2021-01-15T19:09:16.793 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:16.792+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:16.820 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:16.820 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:16.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:16.881 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:16.894 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:16.892+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:16.903 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:16.925 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:16.946 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:16.966 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:16.988 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:16.996 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:16.994+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:17.007 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:17.007 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:17.007 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:17.008 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:17.008 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:17.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:17.034 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:17.034 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.034 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.035 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.035 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.035 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.035 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.036 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.036 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:17.097 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:17.096+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:17.164 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:17.164 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:17.165 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:17.165 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:17.165 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:17.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:17.198 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:17.197+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:17.302 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:17.298+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:17.346 INFO:tasks.rados.rados.0.smithi001.stdout:333: done (7 left) 2021-01-15T19:09:17.347 INFO:tasks.rados.rados.0.smithi001.stdout:334: done (6 left) 2021-01-15T19:09:17.347 INFO:tasks.rados.rados.0.smithi001.stdout:338: done (5 left) 2021-01-15T19:09:17.347 INFO:tasks.rados.rados.0.smithi001.stdout:339: done (4 left) 2021-01-15T19:09:17.348 INFO:tasks.rados.rados.0.smithi001.stdout:342: done (3 left) 2021-01-15T19:09:17.348 INFO:tasks.rados.rados.0.smithi001.stdout:345: done (2 left) 2021-01-15T19:09:17.348 INFO:tasks.rados.rados.0.smithi001.stdout:346: done (1 left) 2021-01-15T19:09:17.348 INFO:tasks.rados.rados.0.smithi001.stdout:348: done (0 left) 2021-01-15T19:09:17.349 INFO:tasks.rados.rados.0.smithi001.stdout:349: setattr oid 28 current snap is 35 2021-01-15T19:09:17.350 INFO:tasks.rados.rados.0.smithi001.stdout:350: read oid 17 snap 32 2021-01-15T19:09:17.350 INFO:tasks.rados.rados.0.smithi001.stdout:350: expect deleted 2021-01-15T19:09:17.351 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 49 current snap is 35 2021-01-15T19:09:17.351 INFO:tasks.rados.rados.0.smithi001.stdout:351: seq_num 144 ranges {2613248=16384} 2021-01-15T19:09:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:marked out osd.5. 2021-01-15T19:09:17.359 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:17.360 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:17.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:17.360 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:17.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.360 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.1s1 55/52 ECSubWrite(tid=100, reqid=osd.6.0:100, at_version=55'37, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:17.361 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.183697+0000", 2021-01-15T19:09:17.361 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.166200691, 2021-01-15T19:09:17.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017634312999999999, 2021-01-15T19:09:17.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:17.363 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183697+0000", 2021-01-15T19:09:17.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183697+0000", 2021-01-15T19:09:17.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6536999999999998e-05 2021-01-15T19:09:17.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183744+0000", 2021-01-15T19:09:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8940000000000002e-05 2021-01-15T19:09:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183763+0000", 2021-01-15T19:09:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4024e-05 2021-01-15T19:09:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183777+0000", 2021-01-15T19:09:17.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2080000000000006e-06 2021-01-15T19:09:17.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183786+0000", 2021-01-15T19:09:17.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4011000000000003e-05 2021-01-15T19:09:17.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183830+0000", 2021-01-15T19:09:17.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3817e-05 2021-01-15T19:09:17.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183844+0000", 2021-01-15T19:09:17.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016034014999999999 2021-01-15T19:09:17.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.199878+0000", 2021-01-15T19:09:17.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001453761 2021-01-15T19:09:17.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.201332+0000", 2021-01-15T19:09:17.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017634312999999999 2021-01-15T19:09:17.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:17.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:17.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.914764187000003, 2021-01-15T19:09:17.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:09:17.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.386 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:09:17.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.387 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:17.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:17.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:09:17.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:09:17.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:09:17.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:09:17.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:09:17.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:09:17.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:17.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:09:17.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:09:17.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:09:17.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:09:17.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:09:17.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:09:17.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:09:17.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:09:17.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:09:17.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:09:17.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:09:17.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:09:17.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:09:17.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:09:17.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:09:17.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:09:17.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:09:17.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:09:17.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:09:17.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:09:17.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:09:17.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:09:17.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.416 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.417 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:17.419 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:17.420 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.914624887000002, 2021-01-15T19:09:17.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:09:17.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.421 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.422 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:09:17.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.422 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:17.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:17.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:09:17.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:09:17.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:17.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:09:17.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:09:17.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:09:17.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:09:17.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:09:17.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:09:17.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:09:17.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:09:17.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:09:17.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:09:17.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:09:17.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:09:17.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:09:17.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:09:17.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:09:17.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:09:17.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:09:17.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:09:17.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:09:17.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:09:17.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:09:17.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:09:17.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:09:17.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:09:17.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:09:17.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:09:17.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.440 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.440 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.441 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:17.441 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:17.441 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.914503781000001, 2021-01-15T19:09:17.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:09:17.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.442 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:09:17.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.443 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:17.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:17.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:09:17.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:09:17.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:09:17.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:09:17.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:09:17.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:09:17.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:09:17.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:09:17.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:09:17.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:09:17.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:09:17.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:09:17.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:09:17.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:09:17.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:09:17.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:09:17.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:09:17.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:09:17.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:09:17.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:09:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:09:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:09:17.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:09:17.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:09:17.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:09:17.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:09:17.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:09:17.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:09:17.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:09:17.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.464 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:17.464 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:17.464 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.144222907, 2021-01-15T19:09:17.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:09:17.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.465 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:09:17.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.466 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:17.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:17.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:09:17.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:09:17.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:09:17.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:09:17.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:09:17.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:09:17.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:09:17.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:09:17.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:09:17.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:09:17.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:09:17.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:09:17.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:09:17.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:09:17.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:09:17.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:09:17.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:09:17.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:09:17.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:09:17.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:09:17.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:09:17.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:09:17.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:09:17.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:09:17.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:09:17.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.135244729, 2021-01-15T19:09:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:09:17.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:09:17.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:17.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:09:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:09:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:09:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:09:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:09:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:09:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:09:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:09:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:09:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:09:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:09:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:09:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:09:17.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:09:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:09:17.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:09:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:09:17.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:09:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:09:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:09:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:09:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:09:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:09:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.525 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:17.526 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.134725029000002, 2021-01-15T19:09:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:09:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:09:17.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:17.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:17.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:09:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:09:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:09:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:09:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:09:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:09:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:09:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:09:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:09:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:09:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:09:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:09:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:09:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:09:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:09:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:09:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:09:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:09:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:09:17.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:09:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:09:17.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:09:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:09:17.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:09:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:09:17.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.133044075000001, 2021-01-15T19:09:17.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:09:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:09:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:09:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:09:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:09:17.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:09:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:09:17.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:09:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:09:17.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:09:17.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:09:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:17.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:09:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:09:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:09:17.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:09:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:09:17.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:09:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:09:17.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:09:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:09:17.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:09:17.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:09:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:09:17.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:09:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:09:17.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:09:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.132911120999999, 2021-01-15T19:09:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:09:17.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:09:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:09:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:09:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:09:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:09:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:09:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:09:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:09:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:09:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:09:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:09:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:09:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:09:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:09:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:09:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:09:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:09:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:09:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:09:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:09:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:09:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:09:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:09:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:09:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:17.586 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.170379851, 2021-01-15T19:09:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:09:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:09:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:09:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:09:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:09:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:09:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:09:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:09:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:09:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:09:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:09:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:09:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:09:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:09:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:09:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:09:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:09:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:09:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:09:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:09:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:09:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:09:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:09:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:09:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:09:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:09:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:09:17.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.603 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.170213258, 2021-01-15T19:09:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:09:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:17.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.605 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:09:17.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.606 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:17.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:17.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:09:17.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:09:17.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:09:17.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:09:17.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:09:17.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:09:17.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:09:17.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:09:17.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:09:17.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:09:17.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:09:17.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:17.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:09:17.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:09:17.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:09:17.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:09:17.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:17.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:09:17.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:09:17.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:09:17.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:09:17.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:17.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:09:17.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:09:17.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:09:17.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:09:17.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:09:17.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:09:17.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:09:17.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:09:17.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.621 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.622 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:17.622 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:17.622 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9716744640000003, 2021-01-15T19:09:17.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:09:17.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:09:17.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.624 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:17.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:17.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:09:17.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:09:17.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:09:17.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:09:17.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:09:17.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:09:17.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:09:17.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:09:17.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:09:17.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:09:17.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:09:17.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:09:17.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:09:17.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:09:17.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:09:17.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:09:17.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:09:17.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:09:17.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:09:17.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:09:17.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:09:17.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:09:17.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:09:17.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:09:17.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:09:17.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.639 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.640 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:17.641 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:17.641 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.969404419, 2021-01-15T19:09:17.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:09:17.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:09:17.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.643 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:17.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:17.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:09:17.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:09:17.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:09:17.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:09:17.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:09:17.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:09:17.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:09:17.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:09:17.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:09:17.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:09:17.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:09:17.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:09:17.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:09:17.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:09:17.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:09:17.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:09:17.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:09:17.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:09:17.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:09:17.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:09:17.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:09:17.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:09:17.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:09:17.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:09:17.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:09:17.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.658 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.660 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:17.660 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:17.660 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.964866228, 2021-01-15T19:09:17.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:09:17.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:09:17.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:17.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:17.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:09:17.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:09:17.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:09:17.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:09:17.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:09:17.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:09:17.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:17.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:09:17.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:09:17.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:09:17.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:09:17.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:09:17.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:09:17.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:09:17.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:09:17.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:09:17.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:09:17.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:09:17.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:09:17.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:09:17.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:09:17.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:09:17.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:09:17.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:09:17.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:09:17.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.677 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.677 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:17.677 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:17.678 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9622803559999999, 2021-01-15T19:09:17.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:09:17.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.679 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.679 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:09:17.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.680 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:17.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:17.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:09:17.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:09:17.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:09:17.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:09:17.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:09:17.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:09:17.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:09:17.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:09:17.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:09:17.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:09:17.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:09:17.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:09:17.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:09:17.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:09:17.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:09:17.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:09:17.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:09:17.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:09:17.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:09:17.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:09:17.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:09:17.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:09:17.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:09:17.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:09:17.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:09:17.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.703 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.704 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:17.705 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:17.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9619060209999999, 2021-01-15T19:09:17.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:09:17.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.707 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:09:17.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.708 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:17.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:17.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:09:17.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:09:17.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:09:17.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:09:17.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:09:17.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:09:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:09:17.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:09:17.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:09:17.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:09:17.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:09:17.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:09:17.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:09:17.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:09:17.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:09:17.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:09:17.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:09:17.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:09:17.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:09:17.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:09:17.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:09:17.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:09:17.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:09:17.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:09:17.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:09:17.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.736 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:17.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:17.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.9617695260000003, 2021-01-15T19:09:17.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:09:17.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:17.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.739 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:09:17.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.740 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:17.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:17.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:09:17.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:09:17.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:09:17.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:09:17.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:09:17.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:09:17.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:09:17.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:09:17.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:09:17.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:09:17.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:09:17.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:17.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:09:17.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:09:17.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:09:17.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:09:17.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:17.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:09:17.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:09:17.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:09:17.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:09:17.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:09:17.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:09:17.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:09:17.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:09:17.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.764 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/84 ECSubWriteReply(tid=160, last_complete=26'11, committed=1, applied=1))", 2021-01-15T19:09:17.764 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.347698+0000", 2021-01-15T19:09:17.765 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.00220062, 2021-01-15T19:09:17.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019258414000000001, 2021-01-15T19:09:17.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:17.766 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347698+0000", 2021-01-15T19:09:17.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347698+0000", 2021-01-15T19:09:17.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6031e-05 2021-01-15T19:09:17.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347744+0000", 2021-01-15T19:09:17.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2021-01-15T19:09:17.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347754+0000", 2021-01-15T19:09:17.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7512000000000001e-05 2021-01-15T19:09:17.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347771+0000", 2021-01-15T19:09:17.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8899999999999996e-06 2021-01-15T19:09:17.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347780+0000", 2021-01-15T19:09:17.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016302565000000001 2021-01-15T19:09:17.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364083+0000", 2021-01-15T19:09:17.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028730869999999999 2021-01-15T19:09:17.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366956+0000", 2021-01-15T19:09:17.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019258414000000001 2021-01-15T19:09:17.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:889 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [copy-from ver 38] snapc 3a=[3a,38,36,2f,2a,26,25,1b,19] ondisk+write+known_if_redirected e86)", 2021-01-15T19:09:17.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:17.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.9794814989999998, 2021-01-15T19:09:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999, 2021-01-15T19:09:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 889 2021-01-15T19:09:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3864e-05 2021-01-15T19:09:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370451+0000", 2021-01-15T19:09:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0156999999999999e-05 2021-01-15T19:09:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370461+0000", 2021-01-15T19:09:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2923e-05 2021-01-15T19:09:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370474+0000", 2021-01-15T19:09:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0399999999999998e-06 2021-01-15T19:09:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370483+0000", 2021-01-15T19:09:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9975999999999994e-05 2021-01-15T19:09:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370573+0000", 2021-01-15T19:09:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001672 2021-01-15T19:09:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370740+0000", 2021-01-15T19:09:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022655103999999999 2021-01-15T19:09:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.393395+0000", 2021-01-15T19:09:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087490950000000001 2021-01-15T19:09:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.402144+0000", 2021-01-15T19:09:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020087099999999999 2021-01-15T19:09:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404153+0000", 2021-01-15T19:09:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0517000000000001e-05 2021-01-15T19:09:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404243+0000", 2021-01-15T19:09:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999 2021-01-15T19:09:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:09:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 2.9793718060000001, 2021-01-15T19:09:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:09:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:09:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:09:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:09:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:09:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:09:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:09:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:17.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:09:17.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:09:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:09:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:09:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:09:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:09:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:09:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:09:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:09:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:09:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:09:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:09:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:09:17.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:09:17.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:09:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:09:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:09:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:09:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:17.810 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:17.812 INFO:tasks.rados.rados.0.smithi001.stdout:351: writing smithi00140323-49 from 2613248 to 2629632 tid 1 2021-01-15T19:09:17.812 INFO:tasks.rados.rados.0.smithi001.stdout:352: read oid 50 snap 25 2021-01-15T19:09:17.812 INFO:tasks.rados.rados.0.smithi001.stdout:352: expect (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:09:17.813 INFO:tasks.rados.rados.0.smithi001.stdout:353: read oid 3 snap -1 2021-01-15T19:09:17.813 INFO:tasks.rados.rados.0.smithi001.stdout:353: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:17.813 INFO:tasks.rados.rados.0.smithi001.stdout:354: rollback oid 43 current snap is 35 2021-01-15T19:09:17.813 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 43 to 25 2021-01-15T19:09:17.813 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 113 (ObjNum 80 snap 14 seq_num 80) dirty exists 2021-01-15T19:09:17.814 INFO:tasks.rados.rados.0.smithi001.stdout:349: done (5 left) 2021-01-15T19:09:17.814 INFO:tasks.rados.rados.0.smithi001.stdout:355: read oid 26 snap -1 2021-01-15T19:09:17.814 INFO:tasks.rados.rados.0.smithi001.stdout:355: expect (ObjNum 103 snap 17 seq_num 103) 2021-01-15T19:09:17.814 INFO:tasks.rados.rados.0.smithi001.stdout:356: snap_remove snap 22 2021-01-15T19:09:17.815 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:17.815 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:17.810+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:17.916 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:17.914+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:17.990 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:17.991 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:17.991 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:17.991 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:18.003 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:18.019 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:18.018+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:18.120 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:18.118+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:18.138 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:18.140 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:18.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:18.187 INFO:tasks.rados.rados.0.smithi001.stdout:356: done (6 left) 2021-01-15T19:09:18.187 INFO:tasks.rados.rados.0.smithi001.stdout:357: read oid 44 snap -1 2021-01-15T19:09:18.187 INFO:tasks.rados.rados.0.smithi001.stdout:357: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:18.188 INFO:tasks.rados.rados.0.smithi001.stdout:354: finishing rollback tid 0 to smithi00140323-43 2021-01-15T19:09:18.188 INFO:tasks.rados.rados.0.smithi001.stdout:351: finishing write tid 1 to smithi00140323-49 2021-01-15T19:09:18.188 INFO:tasks.rados.rados.0.smithi001.stdout:351: finishing write tid 2 to smithi00140323-49 2021-01-15T19:09:18.188 INFO:tasks.rados.rados.0.smithi001.stdout:354: finishing rollback tid 1 to smithi00140323-43 2021-01-15T19:09:18.189 INFO:tasks.rados.rados.0.smithi001.stdout:354: finishing rollback tid 2 to smithi00140323-43 2021-01-15T19:09:18.189 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 117 (ObjNum 42 snap 0 seq_num 42) dirty exists 2021-01-15T19:09:18.189 INFO:tasks.rados.rados.0.smithi001.stdout:351: finishing write tid 3 to smithi00140323-49 2021-01-15T19:09:18.189 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 69 (ObjNum 144 snap 35 seq_num 144) dirty exists 2021-01-15T19:09:18.190 INFO:tasks.rados.rados.0.smithi001.stdout:351: left oid 49 (ObjNum 144 snap 35 seq_num 144) 2021-01-15T19:09:18.190 INFO:tasks.rados.rados.0.smithi001.stdout:352: expect (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:09:18.237 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:18.234+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:18.264 INFO:tasks.rados.rados.0.smithi001.stdout:355: expect (ObjNum 103 snap 17 seq_num 103) 2021-01-15T19:09:18.325 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:18.324+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:18.335 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:18.335 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:18.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:18.336 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:18.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.337 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:18.337 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:18.338 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.083231128999998, 2021-01-15T19:09:18.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:18.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.339 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:18.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.341 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:18.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:18.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:18.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:18.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:18.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:18.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:18.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:18.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:18.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:18.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:18.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:18.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:18.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:18.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:18.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:18.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:18.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:18.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:18.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.360 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.361 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:18.361 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:18.361 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.083040185000002, 2021-01-15T19:09:18.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:18.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:18.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:18.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:18.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:18.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:18.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:18.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:18.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:18.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:18.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:18.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:18.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:18.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:18.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:18.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:18.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:18.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:18.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:18.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:18.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:18.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:18.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:18.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:18.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:18.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:18.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:18.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:18.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:18.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:18.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.377 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.378 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:18.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.144088087999997, 2021-01-15T19:09:18.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:18.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:18.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:18.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:18.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:18.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:18.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:18.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:18.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:18.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:18.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:18.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:18.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:18.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:18.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:18.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:18.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:18.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:18.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:18.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:18.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:18.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:18.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:18.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:18.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:18.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:18.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:18.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.141367627000001, 2021-01-15T19:09:18.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:18.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:18.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:18.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.400 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:18.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:18.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:18.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:18.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:18.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:18.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:18.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:18.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:18.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:18.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:18.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:18.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:18.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:18.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:18.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:18.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:18.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:18.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:18.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:18.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:18.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:18.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:18.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:18.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:18.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:18.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:18.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.415 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.415 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:18.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.141255047999998, 2021-01-15T19:09:18.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:18.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:18.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:18.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:18.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:18.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:18.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:18.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:18.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:18.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:18.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:18.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:18.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:18.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:18.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:18.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:18.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:18.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:18.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:18.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:18.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:18.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:18.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:18.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:18.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:18.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.432 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:18.432 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:18.433 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.141125144, 2021-01-15T19:09:18.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:18.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:18.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:18.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.435 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:18.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:18.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:18.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:18.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:18.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:18.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:18.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:18.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:18.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:18.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:18.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:18.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:18.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:18.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:18.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:18.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:18.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:18.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:18.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:18.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:18.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:18.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:18.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:18.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:18.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:18.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:18.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.450 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.450 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:18.450 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.117919874999998, 2021-01-15T19:09:18.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:18.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:18.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:18.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:18.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:18.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:18.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:18.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:18.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:18.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:18.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:18.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:18.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:18.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:18.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:18.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:18.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:18.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:18.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:18.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:18.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:18.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:18.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.464 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.465 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.465 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:18.465 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.117801677000003, 2021-01-15T19:09:18.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:18.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:18.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.467 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:18.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:18.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:18.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:18.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:18.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:18.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:18.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:18.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:18.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:18.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:18.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:18.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:18.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:18.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:18.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:18.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:18.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:18.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:18.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:18.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:18.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:18.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:18.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.485 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.485 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.486 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:18.486 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.117688987000001, 2021-01-15T19:09:18.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:18.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.487 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:18.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.488 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:18.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:18.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:18.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:18.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:18.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:18.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:18.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:18.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:18.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:18.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:18.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:18.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:18.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:18.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:18.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:18.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:18.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:18.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:18.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:18.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:18.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:18.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:18.501 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.502 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.502 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.503 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:18.503 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:18.503 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.112087236999997, 2021-01-15T19:09:18.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:18.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:18.504 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:18.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:18.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:18.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:18.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:18.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:18.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:18.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:18.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:18.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:18.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:18.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:18.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:18.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:18.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:18.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.513 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.513 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:18.513 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.103618513000001, 2021-01-15T19:09:18.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:18.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.515 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:18.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:18.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:18.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:18.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:18.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:18.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:18.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:18.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:18.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:18.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:18.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:18.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:18.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:18.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:18.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.099099930000001, 2021-01-15T19:09:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:18.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:18.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:18.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:18.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.094692833000003, 2021-01-15T19:09:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.548 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:18.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:18.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:18.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:18.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:18.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:18.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.094210707000002, 2021-01-15T19:09:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:18.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:18.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.586 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.094082161000003, 2021-01-15T19:09:18.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.588 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:18.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:18.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:18.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:18.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.085633021, 2021-01-15T19:09:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:18.618 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.126958083000002, 2021-01-15T19:09:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:18.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:18.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:18.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:18.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:18.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:18.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:18.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:18.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:18.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:18.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:18.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:18.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:18.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:18.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:18.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:18.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:18.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:18.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:18.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:18.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:18.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:18.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:18.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:18.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:18.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.635 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.636 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:18.636 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:18.636 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 16.002773400999999, 2021-01-15T19:09:18.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:18.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:18.637 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:18.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:18.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:18.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:18.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:18.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:18.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:18.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:18.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:18.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:18.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:18.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:18.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:18.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:18.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:18.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:18.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:18.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:18.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:18.648 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:18.648 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.141412112999999, 2021-01-15T19:09:18.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:18.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:18.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:18.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.650 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:18.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:18.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:18.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:18.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:18.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:18.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:18.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:18.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:18.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:18.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:18.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:18.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:18.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:18.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:18.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:18.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:18.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.660 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:18.660 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:18.660 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.149771435, 2021-01-15T19:09:18.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:18.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:18.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:18.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:18.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:18.661 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:18.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:18.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.662 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:18.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:18.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:18.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:18.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:18.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:18.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:18.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:18.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:18.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:18.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:18.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:18.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:18.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:18.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:18.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:18.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:18.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:18.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:18.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:18.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:18.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:18.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:18.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:18.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:18.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:18.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:18.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:18.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:18.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:18.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:18.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:18.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:18.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:18.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:18.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:18.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:18.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:18.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:18.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:18.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:18.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:18.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:18.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:18.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.677 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:18.677 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:18.677 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:18.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:09:18.680 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:18.681 INFO:tasks.rados.rados.0.smithi001.stdout:351: done (6 left) 2021-01-15T19:09:18.681 INFO:tasks.rados.rados.0.smithi001.stdout:352: done (5 left) 2021-01-15T19:09:18.681 INFO:tasks.rados.rados.0.smithi001.stdout:354: done (4 left) 2021-01-15T19:09:18.681 INFO:tasks.rados.rados.0.smithi001.stdout:355: done (3 left) 2021-01-15T19:09:18.682 INFO:tasks.rados.rados.0.smithi001.stdout:358: read oid 8 snap -1 2021-01-15T19:09:18.682 INFO:tasks.rados.rados.0.smithi001.stdout:358: expect (ObjNum 131 snap 31 seq_num 131) 2021-01-15T19:09:18.682 INFO:tasks.rados.rados.0.smithi001.stdout:359: copy_from oid 24 from oid 15 current snap is 35 2021-01-15T19:09:18.682 INFO:tasks.rados.rados.0.smithi001.stdout:360: snap_create 2021-01-15T19:09:18.682 INFO:tasks.rados.rados.0.smithi001.stdout:357: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:18.683 INFO:tasks.rados.rados.0.smithi001.stdout:358: expect (ObjNum 131 snap 31 seq_num 131) 2021-01-15T19:09:18.683 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:18.678+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:18.780 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:18.779+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:18.816 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:18.817 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:18.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:18.909 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:18.908+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:18.982 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:18.981+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:18.992 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:18.992 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:18.993 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:18.994 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:19.002 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:19.043 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:19.087 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:19.086+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:19.170 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:19.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:366 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:19.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.975938147999997, 2021-01-15T19:09:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999, 2021-01-15T19:09:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 366 2021-01-15T19:09:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191899+0000", 2021-01-15T19:09:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0789999999999999e-05 2021-01-15T19:09:19.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191920+0000", 2021-01-15T19:09:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2760000000000001e-06 2021-01-15T19:09:19.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191923+0000", 2021-01-15T19:09:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4840000000000001e-06 2021-01-15T19:09:19.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191930+0000", 2021-01-15T19:09:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2479999999999999e-06 2021-01-15T19:09:19.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.191932+0000", 2021-01-15T19:09:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055448800000000003 2021-01-15T19:09:19.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197477+0000", 2021-01-15T19:09:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101512 2021-01-15T19:09:19.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.197578+0000", 2021-01-15T19:09:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009253308 2021-01-15T19:09:19.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206832+0000", 2021-01-15T19:09:19.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011591699999999999 2021-01-15T19:09:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.206947+0000", 2021-01-15T19:09:19.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012706729999999999 2021-01-15T19:09:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.208218+0000", 2021-01-15T19:09:19.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016319087999999999 2021-01-15T19:09:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.903706858, 2021-01-15T19:09:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:09:19.186 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.189 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:09:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:09:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:09:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:19.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:09:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:09:19.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:09:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:09:19.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:09:19.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:09:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:09:19.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:09:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:09:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:09:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:09:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:09:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:09:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:09:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:09:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:09:19.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:09:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:09:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:19.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.903295231000001, 2021-01-15T19:09:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.206 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.207 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:19.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:19.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:19.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:19.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:693 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:19.227 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.986709757, 2021-01-15T19:09:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668, 2021-01-15T19:09:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 693 2021-01-15T19:09:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2021-01-15T19:09:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181158+0000", 2021-01-15T19:09:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5740000000000001e-06 2021-01-15T19:09:19.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181165+0000", 2021-01-15T19:09:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2294e-05 2021-01-15T19:09:19.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181177+0000", 2021-01-15T19:09:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6979999999999998e-06 2021-01-15T19:09:19.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181185+0000", 2021-01-15T19:09:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.101e-05 2021-01-15T19:09:19.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181246+0000", 2021-01-15T19:09:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2619999999999999e-05 2021-01-15T19:09:19.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181258+0000", 2021-01-15T19:09:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075312950000000004 2021-01-15T19:09:19.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188790+0000", 2021-01-15T19:09:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1936999999999998e-05 2021-01-15T19:09:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188872+0000", 2021-01-15T19:09:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013017509999999999 2021-01-15T19:09:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.201889+0000", 2021-01-15T19:09:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668 2021-01-15T19:09:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:19.242 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.986622729, 2021-01-15T19:09:19.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:19.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:19.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:19.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:19.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:19.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:19.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.986558929000001, 2021-01-15T19:09:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:19.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.282 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:824 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:19.283 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.984618886, 2021-01-15T19:09:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001, 2021-01-15T19:09:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 824 2021-01-15T19:09:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6789000000000003e-05 2021-01-15T19:09:19.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183255+0000", 2021-01-15T19:09:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0429999999999996e-06 2021-01-15T19:09:19.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183264+0000", 2021-01-15T19:09:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7039000000000001e-05 2021-01-15T19:09:19.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183281+0000", 2021-01-15T19:09:19.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0779999999999998e-06 2021-01-15T19:09:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183288+0000", 2021-01-15T19:09:19.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056417399999999997 2021-01-15T19:09:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183852+0000", 2021-01-15T19:09:19.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086462300000000002 2021-01-15T19:09:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184717+0000", 2021-01-15T19:09:19.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019389661999999998 2021-01-15T19:09:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.204107+0000", 2021-01-15T19:09:19.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001 2021-01-15T19:09:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.983976162999999, 2021-01-15T19:09:19.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:19.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:19.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:19.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:19.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:19.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:19.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:19.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:19.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:19.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:19.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:19.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:19.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:19.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:19.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:19.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:19.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:19.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:19.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:19.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.983813222, 2021-01-15T19:09:19.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:19.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.310 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.311 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:19.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.311 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:19.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:19.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:19.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:19.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:19.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:19.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:19.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:19.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:19.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:19.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:19.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:19.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:19.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:19.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:19.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:19.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:19.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:830 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:19.322 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:19.322 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.983692844, 2021-01-15T19:09:19.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018941972000000001, 2021-01-15T19:09:19.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.323 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.324 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 830 2021-01-15T19:09:19.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.324 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:19.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184144+0000", 2021-01-15T19:09:19.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7934e-05 2021-01-15T19:09:19.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184182+0000", 2021-01-15T19:09:19.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8520000000000004e-06 2021-01-15T19:09:19.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184190+0000", 2021-01-15T19:09:19.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2923e-05 2021-01-15T19:09:19.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184203+0000", 2021-01-15T19:09:19.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1710000000000001e-06 2021-01-15T19:09:19.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184208+0000", 2021-01-15T19:09:19.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010232240000000001 2021-01-15T19:09:19.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185231+0000", 2021-01-15T19:09:19.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013339 2021-01-15T19:09:19.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185365+0000", 2021-01-15T19:09:19.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087249099999999998 2021-01-15T19:09:19.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.186237+0000", 2021-01-15T19:09:19.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014278872 2021-01-15T19:09:19.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200516+0000", 2021-01-15T19:09:19.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024695059999999998 2021-01-15T19:09:19.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.202986+0000", 2021-01-15T19:09:19.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100609 2021-01-15T19:09:19.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203086+0000", 2021-01-15T19:09:19.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018941972000000001 2021-01-15T19:09:19.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.346 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:831 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 19] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:19.346 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:19.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.983582887000001, 2021-01-15T19:09:19.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019059996999999999, 2021-01-15T19:09:19.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.349 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 831 2021-01-15T19:09:19.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.350 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:19.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184254+0000", 2021-01-15T19:09:19.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3042e-05 2021-01-15T19:09:19.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184287+0000", 2021-01-15T19:09:19.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8390000000000003e-06 2021-01-15T19:09:19.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184293+0000", 2021-01-15T19:09:19.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0827e-05 2021-01-15T19:09:19.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184304+0000", 2021-01-15T19:09:19.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6419999999999998e-06 2021-01-15T19:09:19.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184309+0000", 2021-01-15T19:09:19.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040504060000000003 2021-01-15T19:09:19.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188359+0000", 2021-01-15T19:09:19.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000122912 2021-01-15T19:09:19.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.188482+0000", 2021-01-15T19:09:19.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066735099999999995 2021-01-15T19:09:19.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189149+0000", 2021-01-15T19:09:19.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01136844 2021-01-15T19:09:19.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200518+0000", 2021-01-15T19:09:19.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027085680000000002 2021-01-15T19:09:19.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203226+0000", 2021-01-15T19:09:19.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.797e-05 2021-01-15T19:09:19.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203314+0000", 2021-01-15T19:09:19.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019059996999999999 2021-01-15T19:09:19.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.369 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.370 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:832 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:19.370 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:19.370 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.983489109000001, 2021-01-15T19:09:19.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596, 2021-01-15T19:09:19.371 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.371 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.372 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 832 2021-01-15T19:09:19.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:19.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:19.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6316000000000003e-05 2021-01-15T19:09:19.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184384+0000", 2021-01-15T19:09:19.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:09:19.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184390+0000", 2021-01-15T19:09:19.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.133e-05 2021-01-15T19:09:19.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184401+0000", 2021-01-15T19:09:19.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7360000000000001e-06 2021-01-15T19:09:19.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184406+0000", 2021-01-15T19:09:19.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054429209999999999 2021-01-15T19:09:19.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189849+0000", 2021-01-15T19:09:19.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013384599999999999 2021-01-15T19:09:19.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189983+0000", 2021-01-15T19:09:19.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053970699999999995 2021-01-15T19:09:19.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.190523+0000", 2021-01-15T19:09:19.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099961230000000009 2021-01-15T19:09:19.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200519+0000", 2021-01-15T19:09:19.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029232770000000002 2021-01-15T19:09:19.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203442+0000", 2021-01-15T19:09:19.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8573999999999996e-05 2021-01-15T19:09:19.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203531+0000", 2021-01-15T19:09:19.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596 2021-01-15T19:09:19.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.7s0 80/52 ECSubRead(tid=294, to_read={3:e0e4bbc9:::smithi00140323-39:head=0,2355200,0}, subchunks={3:e0e4bbc9:::smithi00140323-39:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:19.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:19.386 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.982174937, 2021-01-15T19:09:19.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016692684999999999, 2021-01-15T19:09:19.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:19.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:19.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:19.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:19.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185662+0000", 2021-01-15T19:09:19.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6965000000000001e-05 2021-01-15T19:09:19.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185679+0000", 2021-01-15T19:09:19.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4359999999999996e-06 2021-01-15T19:09:19.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185687+0000", 2021-01-15T19:09:19.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011445029000000001 2021-01-15T19:09:19.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.197132+0000", 2021-01-15T19:09:19.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052232550000000004 2021-01-15T19:09:19.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.202355+0000", 2021-01-15T19:09:19.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016692684999999999 2021-01-15T19:09:19.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.395 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.396 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:19.396 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:19.396 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.8227705759999999, 2021-01-15T19:09:19.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:19.396 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:19.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.398 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:19.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:19.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:19.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:19.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:19.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:19.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:19.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:19.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:19.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:19.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:19.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:19.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:19.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:19.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:19.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:19.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:19.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:19.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:19.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:19.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:19.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:19.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:19.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:19.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:19.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:19.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:19.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:19.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.413 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.414 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:19.414 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:19.414 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.8226631199999996, 2021-01-15T19:09:19.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:19.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.416 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:19.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.417 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:19.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.418 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:19.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:19.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:19.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:19.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:19.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:19.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:19.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:19.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:19.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:19.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:19.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:19.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:19.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:19.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:19.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:19.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:19.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:19.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:19.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:19.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:19.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:19.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:19.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:19.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:19.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:19.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:19.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:19.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:19.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:19.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:19.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:19.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:19.439 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:19.439 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.6895962539999996, 2021-01-15T19:09:19.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:19.439 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.439 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.440 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:19.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:19.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:19.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:19.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:19.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:19.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:19.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:19.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:19.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:19.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:19.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:19.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:19.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:19.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:19.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:19.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:19.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:19.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:19.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:19.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:19.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:19.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:19.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:19.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:19.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:19.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:19.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:19.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:19.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.462 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:878 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [delete] snapc 3a=[3a,38,36,2f,2a,26,25,1b,19] ondisk+write+peerstat_old+known_if_redirected e86)", 2021-01-15T19:09:19.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:14.333165+0000", 2021-01-15T19:09:19.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8346716809999997, 2021-01-15T19:09:19.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017469419999999999, 2021-01-15T19:09:19.464 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.464 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:19.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.465 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 878 2021-01-15T19:09:19.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.475 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333165+0000", 2021-01-15T19:09:19.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333165+0000", 2021-01-15T19:09:19.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8362000000000003e-05 2021-01-15T19:09:19.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333204+0000", 2021-01-15T19:09:19.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6459999999999999e-06 2021-01-15T19:09:19.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333213+0000", 2021-01-15T19:09:19.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5153e-05 2021-01-15T19:09:19.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333228+0000", 2021-01-15T19:09:19.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5009999999999998e-06 2021-01-15T19:09:19.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333235+0000", 2021-01-15T19:09:19.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1539999999999997e-05 2021-01-15T19:09:19.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333297+0000", 2021-01-15T19:09:19.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2990000000000002e-06 2021-01-15T19:09:19.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:19.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.333306+0000", 2021-01-15T19:09:19.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0088916129999999996 2021-01-15T19:09:19.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.342198+0000", 2021-01-15T19:09:19.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015474400000000001 2021-01-15T19:09:19.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.342352+0000", 2021-01-15T19:09:19.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062307300000000003 2021-01-15T19:09:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.342975+0000", 2021-01-15T19:09:19.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038617909999999998 2021-01-15T19:09:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.346837+0000", 2021-01-15T19:09:19.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036997660000000002 2021-01-15T19:09:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.350537+0000", 2021-01-15T19:09:19.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7931999999999997e-05 2021-01-15T19:09:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:14.350635+0000", 2021-01-15T19:09:19.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017469419999999999 2021-01-15T19:09:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:19.518 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:19.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7936285709999999, 2021-01-15T19:09:19.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:19.519 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.519 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:19.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.520 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:19.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.521 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:19.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:19.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:19.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:19.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:19.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:19.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:19.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:19.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:19.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:19.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:19.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:19.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:19.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:19.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:19.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:19.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:19.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:19.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:19.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:19.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:19.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:19.536 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7934984410000001, 2021-01-15T19:09:19.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:19.536 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:19.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.537 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:19.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.538 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:19.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:19.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:19.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:19.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:19.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:19.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:19.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:19.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:19.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:19.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:19.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:19.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:19.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:19.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:19.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:19.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:19.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:19.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:19.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:19.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.551 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:19.551 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:19.551 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7934130319999999, 2021-01-15T19:09:19.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:19.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:19.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:19.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:19.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:19.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:19.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:19.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:19.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:19.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:19.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:19.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:19.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:19.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:19.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:19.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:19.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:19.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:19.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:19.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:19.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:19.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:19.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:19.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:19.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:19.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:19.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:19.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:19.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:19.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:19.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:19.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:19.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:19.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:19.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:19.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:19.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:19.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:19.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:19.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.564 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:19.565 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:19.565 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:19.568 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:19.569 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:19.569 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:19.569 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:19.569 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:19.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:19.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:19.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:09:19.571 INFO:tasks.rados.rados.0.smithi001.stdout:357: done (5 left) 2021-01-15T19:09:19.571 INFO:tasks.rados.rados.0.smithi001.stdout:358: done (4 left) 2021-01-15T19:09:19.571 INFO:tasks.rados.rados.0.smithi001.stdout:360: done (3 left) 2021-01-15T19:09:19.571 INFO:tasks.rados.rados.0.smithi001.stdout:361: rmattr oid 1 current snap is 36 2021-01-15T19:09:19.572 INFO:tasks.rados.rados.0.smithi001.stdout:361: done (3 left) 2021-01-15T19:09:19.572 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 5 current snap is 36 2021-01-15T19:09:19.572 INFO:tasks.rados.rados.0.smithi001.stdout:362: seq_num 145 ranges {319488=32768} 2021-01-15T19:09:19.572 INFO:tasks.rados.rados.0.smithi001.stdout:362: writing smithi00140323-5 from 319488 to 352256 tid 1 2021-01-15T19:09:19.572 INFO:tasks.rados.rados.0.smithi001.stdout:363: read oid 45 snap -1 2021-01-15T19:09:19.573 INFO:tasks.rados.rados.0.smithi001.stdout:363: expect (ObjNum 89 snap 16 seq_num 89) 2021-01-15T19:09:19.573 INFO:tasks.rados.rados.0.smithi001.stdout:364: snap_create 2021-01-15T19:09:19.573 INFO:tasks.rados.rados.0.smithi001.stdout:363: expect (ObjNum 89 snap 16 seq_num 89) 2021-01-15T19:09:19.573 INFO:tasks.rados.rados.0.smithi001.stdout:353: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:19.574 INFO:tasks.rados.rados.0.smithi001.stdout:359: finishing copy_from to smithi00140323-24 2021-01-15T19:09:19.574 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 24 v 54 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:19.574 INFO:tasks.rados.rados.0.smithi001.stdout:359: finishing copy_from racing read to smithi00140323-24 2021-01-15T19:09:19.574 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:19.576 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:19.565+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:19.577 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:19.668 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:19.667+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:19.741 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:19.741 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:19.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:19.792 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:19.791+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:19.869 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:19.868+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:19.871 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:19.872 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:19.881 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:19.969 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:19.968+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:20.046 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:20.047 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:20.056 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:09:20.097 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:20.096+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:20.171 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:20.171+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:20.195 INFO:tasks.rados.rados.0.smithi001.stdout:350: done (5 left) 2021-01-15T19:09:20.196 INFO:tasks.rados.rados.0.smithi001.stdout:353: done (4 left) 2021-01-15T19:09:20.196 INFO:tasks.rados.rados.0.smithi001.stdout:359: done (3 left) 2021-01-15T19:09:20.196 INFO:tasks.rados.rados.0.smithi001.stdout:363: done (2 left) 2021-01-15T19:09:20.196 INFO:tasks.rados.rados.0.smithi001.stdout:364: done (1 left) 2021-01-15T19:09:20.197 INFO:tasks.rados.rados.0.smithi001.stdout:365: snap_remove snap 35 2021-01-15T19:09:20.228 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:20.228 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:20.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:09:20.282 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:20.281+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:20.364 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:20.365 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:20.373 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:20.372+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:20.374 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:09:20.474 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:20.473+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:20.540 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:20.541 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:20.550 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:20.575 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:20.574+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:20.676 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:20.675+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:20.676 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:20.677 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:20.686 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:20.777 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:20.776+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:20.852 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:20.853 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:20.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:20.879 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:20.877+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:20.980 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:20.979+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:21.028 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:21.028 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:21.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:21.082 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:21.080+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:21.182 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:21.181+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:21.183 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:21.183 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:21.195 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:21.200 INFO:tasks.rados.rados.0.smithi001.stdout:365: done (1 left) 2021-01-15T19:09:21.200 INFO:tasks.rados.rados.0.smithi001.stdout:366: copy_from oid 21 from oid 16 current snap is 37 2021-01-15T19:09:21.200 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 37 current snap is 37 2021-01-15T19:09:21.200 INFO:tasks.rados.rados.0.smithi001.stdout:367: seq_num 146 ranges {4456448=49152} 2021-01-15T19:09:21.226 INFO:tasks.rados.rados.0.smithi001.stdout:367: writing smithi00140323-37 from 4456448 to 4505600 tid 1 2021-01-15T19:09:21.226 INFO:tasks.rados.rados.0.smithi001.stdout:368: snap_create 2021-01-15T19:09:21.227 INFO:tasks.rados.rados.0.smithi001.stdout:366: finishing copy_from to smithi00140323-21 2021-01-15T19:09:21.227 INFO:tasks.rados.rados.0.smithi001.stdout:366: got expected ENOENT (src dne) 2021-01-15T19:09:21.228 INFO:tasks.rados.rados.0.smithi001.stdout:366: finishing copy_from racing read to smithi00140323-21 2021-01-15T19:09:21.232 INFO:tasks.rados.rados.0.smithi001.stdout:367: finishing write tid 1 to smithi00140323-37 2021-01-15T19:09:21.233 INFO:tasks.rados.rados.0.smithi001.stdout:367: finishing write tid 2 to smithi00140323-37 2021-01-15T19:09:21.236 INFO:tasks.rados.rados.0.smithi001.stdout:367: finishing write tid 3 to smithi00140323-37 2021-01-15T19:09:21.236 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 71 (ObjNum 146 snap 37 seq_num 146) dirty exists 2021-01-15T19:09:21.236 INFO:tasks.rados.rados.0.smithi001.stdout:367: left oid 37 (ObjNum 146 snap 37 seq_num 146) 2021-01-15T19:09:21.283 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:21.282+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:21.339 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:21.339 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:21.349 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:21.384 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:21.383+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:21.484 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:21.483+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:21.521 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:21.521 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:21.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:21.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:09:21.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:21.585 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:21.585+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:21.688 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:21.686+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:21.707 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:21.708 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:21.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:21.777 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:21.787 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:21.787+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:21.797 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:21.818 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:21.839 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:21.860 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:21.861 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:21.862 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:21.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:21.882 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:21.888 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:21.887+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:21.905 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:09:21.926 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:21.988 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:21.988+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:22.010 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.011 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:09:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:09:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.491235849999995, 2021-01-15T19:09:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:09:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:09:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:09:22.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:09:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:09:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:09:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:09:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:09:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:09:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:09:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:09:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:09:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:09:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:09:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:09:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:09:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:09:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:09:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:09:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:09:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:09:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:09:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:09:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.024 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.fs1 55/21 ECSubWrite(tid=128, reqid=client.4418.0:497, at_version=55'34, trim_to=0'0, roll_forward_to=55'32))", 2021-01-15T19:09:22.025 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.209612+0000", 2021-01-15T19:09:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.798478547999999, 2021-01-15T19:09:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334580000000007, 2021-01-15T19:09:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209612+0000", 2021-01-15T19:09:22.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209612+0000", 2021-01-15T19:09:22.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000100581 2021-01-15T19:09:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209712+0000", 2021-01-15T19:09:22.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7216000000000002e-05 2021-01-15T19:09:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209740+0000", 2021-01-15T19:09:22.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7572999999999999e-05 2021-01-15T19:09:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209757+0000", 2021-01-15T19:09:22.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209765+0000", 2021-01-15T19:09:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7841999999999999e-05 2021-01-15T19:09:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209823+0000", 2021-01-15T19:09:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7425e-05 2021-01-15T19:09:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.209841+0000", 2021-01-15T19:09:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0082645830000000007 2021-01-15T19:09:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.218105+0000", 2021-01-15T19:09:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0083999999999998e-05 2021-01-15T19:09:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.218145+0000", 2021-01-15T19:09:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334580000000007 2021-01-15T19:09:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/21 ECSubWrite(tid=170, reqid=client.4418.0:550, at_version=56'27, trim_to=0'0, roll_forward_to=55'25))", 2021-01-15T19:09:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.207900+0000", 2021-01-15T19:09:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.800189904, 2021-01-15T19:09:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0088003890000000005, 2021-01-15T19:09:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.207900+0000", 2021-01-15T19:09:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.207900+0000", 2021-01-15T19:09:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015956999999999999 2021-01-15T19:09:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.208060+0000", 2021-01-15T19:09:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5885999999999999e-05 2021-01-15T19:09:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.208086+0000", 2021-01-15T19:09:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5574999999999999e-05 2021-01-15T19:09:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.208101+0000", 2021-01-15T19:09:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.793e-06 2021-01-15T19:09:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.208108+0000", 2021-01-15T19:09:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6329999999999999e-05 2021-01-15T19:09:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.208155+0000", 2021-01-15T19:09:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2405999999999999e-05 2021-01-15T19:09:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.208167+0000", 2021-01-15T19:09:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0084812710000000003 2021-01-15T19:09:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216648+0000", 2021-01-15T19:09:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2558000000000001e-05 2021-01-15T19:09:22.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216701+0000", 2021-01-15T19:09:22.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0088003890000000005 2021-01-15T19:09:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 56/21 ECSubWrite(tid=171, reqid=client.4418.0:564, at_version=56'49, trim_to=0'0, roll_forward_to=55'47))", 2021-01-15T19:09:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216127+0000", 2021-01-15T19:09:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.791962949000002, 2021-01-15T19:09:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010909558, 2021-01-15T19:09:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216127+0000", 2021-01-15T19:09:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216127+0000", 2021-01-15T19:09:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0769000000000001e-05 2021-01-15T19:09:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216178+0000", 2021-01-15T19:09:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7861999999999999e-05 2021-01-15T19:09:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216206+0000", 2021-01-15T19:09:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7056000000000001e-05 2021-01-15T19:09:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216223+0000", 2021-01-15T19:09:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3429999999999998e-06 2021-01-15T19:09:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216230+0000", 2021-01-15T19:09:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8094999999999999e-05 2021-01-15T19:09:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216289+0000", 2021-01-15T19:09:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2507000000000001e-05 2021-01-15T19:09:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216301+0000", 2021-01-15T19:09:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0106803 2021-01-15T19:09:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226981+0000", 2021-01-15T19:09:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5625999999999999e-05 2021-01-15T19:09:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.227037+0000", 2021-01-15T19:09:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010909558 2021-01-15T19:09:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 56/21 ECSubWrite(tid=172, reqid=client.4418.0:565, at_version=56'50, trim_to=0'0, roll_forward_to=55'47))", 2021-01-15T19:09:22.063 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.232219+0000", 2021-01-15T19:09:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.775871160000001, 2021-01-15T19:09:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011728980999999999, 2021-01-15T19:09:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.064 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232219+0000", 2021-01-15T19:09:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232219+0000", 2021-01-15T19:09:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8811999999999999e-05 2021-01-15T19:09:22.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232278+0000", 2021-01-15T19:09:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5379000000000001e-05 2021-01-15T19:09:22.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232313+0000", 2021-01-15T19:09:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4078000000000001e-05 2021-01-15T19:09:22.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232347+0000", 2021-01-15T19:09:22.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2938999999999999e-05 2021-01-15T19:09:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232360+0000", 2021-01-15T19:09:22.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0226000000000002e-05 2021-01-15T19:09:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232421+0000", 2021-01-15T19:09:22.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5248e-05 2021-01-15T19:09:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232436+0000", 2021-01-15T19:09:22.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011465718999999999 2021-01-15T19:09:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243902+0000", 2021-01-15T19:09:22.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6579999999999998e-05 2021-01-15T19:09:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243948+0000", 2021-01-15T19:09:22.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011728980999999999 2021-01-15T19:09:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.ds2 56/21 ECSubWrite(tid=173, reqid=client.4418.0:550, at_version=0'0, trim_to=0'0, roll_forward_to=56'27))", 2021-01-15T19:09:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.232429+0000", 2021-01-15T19:09:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.775661476000003, 2021-01-15T19:09:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010982554, 2021-01-15T19:09:22.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232429+0000", 2021-01-15T19:09:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232429+0000", 2021-01-15T19:09:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0870000000000004e-05 2021-01-15T19:09:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232500+0000", 2021-01-15T19:09:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5313000000000001e-05 2021-01-15T19:09:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232515+0000", 2021-01-15T19:09:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.428e-05 2021-01-15T19:09:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232529+0000", 2021-01-15T19:09:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1970000000000004e-06 2021-01-15T19:09:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232535+0000", 2021-01-15T19:09:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1084000000000001e-05 2021-01-15T19:09:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232586+0000", 2021-01-15T19:09:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2469e-05 2021-01-15T19:09:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.232598+0000", 2021-01-15T19:09:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010756528 2021-01-15T19:09:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243355+0000", 2021-01-15T19:09:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6813e-05 2021-01-15T19:09:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243411+0000", 2021-01-15T19:09:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010982554 2021-01-15T19:09:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.087 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 56/21 ECSubWrite(tid=176, reqid=client.4418.0:552, at_version=56'52, trim_to=0'0, roll_forward_to=55'47))", 2021-01-15T19:09:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.247432+0000", 2021-01-15T19:09:22.088 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.760658669000001, 2021-01-15T19:09:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010545913000000001, 2021-01-15T19:09:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247432+0000", 2021-01-15T19:09:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247432+0000", 2021-01-15T19:09:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00059555299999999997 2021-01-15T19:09:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248027+0000", 2021-01-15T19:09:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7098999999999999e-05 2021-01-15T19:09:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248054+0000", 2021-01-15T19:09:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0002e-05 2021-01-15T19:09:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248064+0000", 2021-01-15T19:09:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9089999999999997e-06 2021-01-15T19:09:22.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248070+0000", 2021-01-15T19:09:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3214000000000002e-05 2021-01-15T19:09:22.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248123+0000", 2021-01-15T19:09:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3807e-05 2021-01-15T19:09:22.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248137+0000", 2021-01-15T19:09:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097826960000000004 2021-01-15T19:09:22.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.257920+0000", 2021-01-15T19:09:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7633000000000002e-05 2021-01-15T19:09:22.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.257978+0000", 2021-01-15T19:09:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010545913000000001 2021-01-15T19:09:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.100 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 56/21 ECSubWrite(tid=182, reqid=client.4418.0:566, at_version=0'0, trim_to=0'0, roll_forward_to=56'53))", 2021-01-15T19:09:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.281543+0000", 2021-01-15T19:09:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.726547050000001, 2021-01-15T19:09:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083373890000000006, 2021-01-15T19:09:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281543+0000", 2021-01-15T19:09:22.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281543+0000", 2021-01-15T19:09:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2774999999999994e-05 2021-01-15T19:09:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281606+0000", 2021-01-15T19:09:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8408000000000001e-05 2021-01-15T19:09:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281624+0000", 2021-01-15T19:09:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0239000000000001e-05 2021-01-15T19:09:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281645+0000", 2021-01-15T19:09:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5519999999999993e-06 2021-01-15T19:09:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281654+0000", 2021-01-15T19:09:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013076900000000001 2021-01-15T19:09:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281785+0000", 2021-01-15T19:09:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4779e-05 2021-01-15T19:09:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281800+0000", 2021-01-15T19:09:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0077546380000000003 2021-01-15T19:09:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.289554+0000", 2021-01-15T19:09:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032622899999999998 2021-01-15T19:09:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.289881+0000", 2021-01-15T19:09:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083373890000000006 2021-01-15T19:09:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 56/55 ECSubWrite(tid=80, reqid=client.4418.0:554, at_version=56'37, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.296601+0000", 2021-01-15T19:09:22.113 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.711488869999997, 2021-01-15T19:09:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073909329999999997, 2021-01-15T19:09:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296601+0000", 2021-01-15T19:09:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296601+0000", 2021-01-15T19:09:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015169599999999999 2021-01-15T19:09:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296753+0000", 2021-01-15T19:09:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0713999999999999e-05 2021-01-15T19:09:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296774+0000", 2021-01-15T19:09:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1268e-05 2021-01-15T19:09:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296785+0000", 2021-01-15T19:09:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1550000000000004e-06 2021-01-15T19:09:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296791+0000", 2021-01-15T19:09:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3854000000000003e-05 2021-01-15T19:09:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296845+0000", 2021-01-15T19:09:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7753000000000001e-05 2021-01-15T19:09:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296863+0000", 2021-01-15T19:09:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0070730990000000002 2021-01-15T19:09:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303936+0000", 2021-01-15T19:09:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6394000000000003e-05 2021-01-15T19:09:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303992+0000", 2021-01-15T19:09:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073909329999999997 2021-01-15T19:09:22.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.125 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:631 3.10s0 3:0c50693c:::smithi00140323-16:head [stat,delete] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.180880+0000", 2021-01-15T19:09:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.827210158, 2021-01-15T19:09:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010486093, 2021-01-15T19:09:22.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 631 2021-01-15T19:09:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.180880+0000", 2021-01-15T19:09:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.180880+0000", 2021-01-15T19:09:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.333e-05 2021-01-15T19:09:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.180924+0000", 2021-01-15T19:09:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2337999999999999e-05 2021-01-15T19:09:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.180936+0000", 2021-01-15T19:09:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7180000000000002e-05 2021-01-15T19:09:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.180953+0000", 2021-01-15T19:09:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4221000000000001e-05 2021-01-15T19:09:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.180977+0000", 2021-01-15T19:09:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2139999999999995e-05 2021-01-15T19:09:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.181069+0000", 2021-01-15T19:09:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3643e-05 2021-01-15T19:09:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.181083+0000", 2021-01-15T19:09:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0062377919999999998 2021-01-15T19:09:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.187321+0000", 2021-01-15T19:09:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017300799999999999 2021-01-15T19:09:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.187494+0000", 2021-01-15T19:09:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00088469199999999997 2021-01-15T19:09:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188379+0000", 2021-01-15T19:09:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0016100660000000001 2021-01-15T19:09:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.189989+0000", 2021-01-15T19:09:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013201269999999999 2021-01-15T19:09:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.191309+0000", 2021-01-15T19:09:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7556000000000003e-05 2021-01-15T19:09:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.191366+0000", 2021-01-15T19:09:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010486093 2021-01-15T19:09:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.142 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.fs1 79/21 ECSubWrite(tid=220, reqid=client.4418.0:814, at_version=79'43, trim_to=0'0, roll_forward_to=71'41))", 2021-01-15T19:09:22.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.356211+0000", 2021-01-15T19:09:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.651879793999999, 2021-01-15T19:09:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011041497000000001, 2021-01-15T19:09:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.356211+0000", 2021-01-15T19:09:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.356211+0000", 2021-01-15T19:09:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077664100000000001 2021-01-15T19:09:22.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.356987+0000", 2021-01-15T19:09:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8152999999999998e-05 2021-01-15T19:09:22.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.357035+0000", 2021-01-15T19:09:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0259e-05 2021-01-15T19:09:22.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.357056+0000", 2021-01-15T19:09:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3229999999999998e-06 2021-01-15T19:09:22.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.357064+0000", 2021-01-15T19:09:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7067e-05 2021-01-15T19:09:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.357131+0000", 2021-01-15T19:09:22.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7963999999999999e-05 2021-01-15T19:09:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.357149+0000", 2021-01-15T19:09:22.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010045093 2021-01-15T19:09:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.367194+0000", 2021-01-15T19:09:22.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7997000000000001e-05 2021-01-15T19:09:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.367252+0000", 2021-01-15T19:09:22.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011041497000000001 2021-01-15T19:09:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:827 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182297+0000", 2021-01-15T19:09:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.825793031, 2021-01-15T19:09:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011167837, 2021-01-15T19:09:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 827 2021-01-15T19:09:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182297+0000", 2021-01-15T19:09:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182297+0000", 2021-01-15T19:09:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4666999999999993e-05 2021-01-15T19:09:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182392+0000", 2021-01-15T19:09:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4182000000000001e-05 2021-01-15T19:09:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1441999999999999e-05 2021-01-15T19:09:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.346e-05 2021-01-15T19:09:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4982999999999996e-05 2021-01-15T19:09:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182526+0000", 2021-01-15T19:09:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1535999999999999e-05 2021-01-15T19:09:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182538+0000", 2021-01-15T19:09:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002629485 2021-01-15T19:09:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185167+0000", 2021-01-15T19:09:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143499 2021-01-15T19:09:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185311+0000", 2021-01-15T19:09:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.008154583 2021-01-15T19:09:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.193465+0000", 2021-01-15T19:09:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011167837 2021-01-15T19:09:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.169 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:828 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:22.170 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:09:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.825595304, 2021-01-15T19:09:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011908975, 2021-01-15T19:09:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:22.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 828 2021-01-15T19:09:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.172 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:09:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:09:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9705000000000001e-05 2021-01-15T19:09:22.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182525+0000", 2021-01-15T19:09:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6079999999999998e-06 2021-01-15T19:09:22.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182529+0000", 2021-01-15T19:09:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9899999999999997e-06 2021-01-15T19:09:22.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182537+0000", 2021-01-15T19:09:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1290000000000004e-06 2021-01-15T19:09:22.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182542+0000", 2021-01-15T19:09:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2103999999999999e-05 2021-01-15T19:09:22.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182615+0000", 2021-01-15T19:09:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0329999999999992e-06 2021-01-15T19:09:22.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:09:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040604489999999998 2021-01-15T19:09:22.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.186683+0000", 2021-01-15T19:09:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116184 2021-01-15T19:09:22.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.186799+0000", 2021-01-15T19:09:22.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0076047729999999996 2021-01-15T19:09:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.194404+0000", 2021-01-15T19:09:22.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011908975 2021-01-15T19:09:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:829 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:09:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.82552153, 2021-01-15T19:09:22.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001, 2021-01-15T19:09:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 829 2021-01-15T19:09:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:09:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:09:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4360000000000001e-05 2021-01-15T19:09:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:09:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0169999999999999e-06 2021-01-15T19:09:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182630+0000", 2021-01-15T19:09:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4198e-05 2021-01-15T19:09:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182644+0000", 2021-01-15T19:09:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1253999999999999e-05 2021-01-15T19:09:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182656+0000", 2021-01-15T19:09:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9756999999999999e-05 2021-01-15T19:09:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182695+0000", 2021-01-15T19:09:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2819999999999996e-06 2021-01-15T19:09:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182704+0000", 2021-01-15T19:09:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048051730000000003 2021-01-15T19:09:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187509+0000", 2021-01-15T19:09:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114002 2021-01-15T19:09:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187623+0000", 2021-01-15T19:09:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012123558 2021-01-15T19:09:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.199746+0000", 2021-01-15T19:09:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001 2021-01-15T19:09:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.198 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=315, reqid=client.4418.0:833, at_version=80'84, trim_to=0'0, roll_forward_to=78'82))", 2021-01-15T19:09:22.199 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:09:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.820959563000001, 2021-01-15T19:09:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001, 2021-01-15T19:09:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:09:22.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:09:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6477000000000002e-05 2021-01-15T19:09:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187187+0000", 2021-01-15T19:09:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4749e-05 2021-01-15T19:09:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187212+0000", 2021-01-15T19:09:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5977e-05 2021-01-15T19:09:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187228+0000", 2021-01-15T19:09:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0770000000000008e-06 2021-01-15T19:09:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187236+0000", 2021-01-15T19:09:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014010800000000001 2021-01-15T19:09:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187376+0000", 2021-01-15T19:09:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8624999999999999e-05 2021-01-15T19:09:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187395+0000", 2021-01-15T19:09:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013611689 2021-01-15T19:09:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201006+0000", 2021-01-15T19:09:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7980999999999999e-05 2021-01-15T19:09:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201064+0000", 2021-01-15T19:09:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001 2021-01-15T19:09:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.6s2 80/52 ECSubWrite(tid=293, reqid=osd.4.0:293, at_version=80'56, trim_to=0'0, roll_forward_to=71'54))", 2021-01-15T19:09:22.212 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.190231+0000", 2021-01-15T19:09:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.817858920999999, 2021-01-15T19:09:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075189580000000001, 2021-01-15T19:09:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.213 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190231+0000", 2021-01-15T19:09:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190231+0000", 2021-01-15T19:09:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7855000000000001e-05 2021-01-15T19:09:22.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190289+0000", 2021-01-15T19:09:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.1516000000000001e-05 2021-01-15T19:09:22.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190321+0000", 2021-01-15T19:09:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6626999999999999e-05 2021-01-15T19:09:22.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190337+0000", 2021-01-15T19:09:22.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5210000000000002e-06 2021-01-15T19:09:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190346+0000", 2021-01-15T19:09:22.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000216624 2021-01-15T19:09:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190563+0000", 2021-01-15T19:09:22.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9009999999999999e-05 2021-01-15T19:09:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.190582+0000", 2021-01-15T19:09:22.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0071145770000000004 2021-01-15T19:09:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.197696+0000", 2021-01-15T19:09:22.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4228000000000003e-05 2021-01-15T19:09:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.197750+0000", 2021-01-15T19:09:22.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0075189580000000001 2021-01-15T19:09:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=319, reqid=client.4418.0:833, at_version=0'0, trim_to=0'0, roll_forward_to=80'84))", 2021-01-15T19:09:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.204769+0000", 2021-01-15T19:09:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 13.803321462, 2021-01-15T19:09:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007414866, 2021-01-15T19:09:22.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204769+0000", 2021-01-15T19:09:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204769+0000", 2021-01-15T19:09:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5998999999999999e-05 2021-01-15T19:09:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204805+0000", 2021-01-15T19:09:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.859e-06 2021-01-15T19:09:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204815+0000", 2021-01-15T19:09:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0043e-05 2021-01-15T19:09:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204825+0000", 2021-01-15T19:09:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7659999999999998e-06 2021-01-15T19:09:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204830+0000", 2021-01-15T19:09:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5175e-05 2021-01-15T19:09:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204875+0000", 2021-01-15T19:09:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3067e-05 2021-01-15T19:09:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.204888+0000", 2021-01-15T19:09:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072562269999999996 2021-01-15T19:09:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.212144+0000", 2021-01-15T19:09:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9730000000000001e-05 2021-01-15T19:09:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.212184+0000", 2021-01-15T19:09:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.007414866 2021-01-15T19:09:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.7s2 85/52 ECSubWrite(tid=325, reqid=client.4418.0:847, at_version=85'41, trim_to=0'0, roll_forward_to=69'39))", 2021-01-15T19:09:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.381343+0000", 2021-01-15T19:09:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.6267473510000006, 2021-01-15T19:09:22.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081214170000000006, 2021-01-15T19:09:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381343+0000", 2021-01-15T19:09:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381343+0000", 2021-01-15T19:09:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3838000000000002e-05 2021-01-15T19:09:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381397+0000", 2021-01-15T19:09:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7911999999999999e-05 2021-01-15T19:09:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381435+0000", 2021-01-15T19:09:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6535e-05 2021-01-15T19:09:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381451+0000", 2021-01-15T19:09:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6639999999999993e-06 2021-01-15T19:09:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381461+0000", 2021-01-15T19:09:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0138999999999999e-05 2021-01-15T19:09:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381551+0000", 2021-01-15T19:09:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6416999999999999e-05 2021-01-15T19:09:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381568+0000", 2021-01-15T19:09:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0078373200000000001 2021-01-15T19:09:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.389405+0000", 2021-01-15T19:09:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9592000000000001e-05 2021-01-15T19:09:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.389464+0000", 2021-01-15T19:09:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081214170000000006 2021-01-15T19:09:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.248 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 86/21 ECSubWrite(tid=342, reqid=client.4418.0:888, at_version=86'100, trim_to=0'0, roll_forward_to=86'98))", 2021-01-15T19:09:22.249 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.371874+0000", 2021-01-15T19:09:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.6362159859999998, 2021-01-15T19:09:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0084555840000000004, 2021-01-15T19:09:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.250 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371874+0000", 2021-01-15T19:09:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371874+0000", 2021-01-15T19:09:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110308 2021-01-15T19:09:22.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371985+0000", 2021-01-15T19:09:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1879999999999999e-05 2021-01-15T19:09:22.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.372027+0000", 2021-01-15T19:09:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9758999999999998e-05 2021-01-15T19:09:22.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.372046+0000", 2021-01-15T19:09:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8969999999999997e-06 2021-01-15T19:09:22.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.372055+0000", 2021-01-15T19:09:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6787999999999996e-05 2021-01-15T19:09:22.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.372152+0000", 2021-01-15T19:09:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5996000000000001e-05 2021-01-15T19:09:22.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.372168+0000", 2021-01-15T19:09:22.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0081123919999999995 2021-01-15T19:09:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.380280+0000", 2021-01-15T19:09:22.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9564000000000002e-05 2021-01-15T19:09:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.380330+0000", 2021-01-15T19:09:22.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0084555840000000004 2021-01-15T19:09:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 90/21 ECSubWrite(tid=348, reqid=osd.3.0:348, at_version=90'108, trim_to=0'0, roll_forward_to=88'104))", 2021-01-15T19:09:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:18.193008+0000", 2021-01-15T19:09:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.8150820479999998, 2021-01-15T19:09:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0084513550000000007, 2021-01-15T19:09:22.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193008+0000", 2021-01-15T19:09:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193008+0000", 2021-01-15T19:09:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6971000000000001e-05 2021-01-15T19:09:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193065+0000", 2021-01-15T19:09:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0687e-05 2021-01-15T19:09:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193086+0000", 2021-01-15T19:09:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4331000000000001e-05 2021-01-15T19:09:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193100+0000", 2021-01-15T19:09:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.964e-06 2021-01-15T19:09:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193106+0000", 2021-01-15T19:09:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036775099999999998 2021-01-15T19:09:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193474+0000", 2021-01-15T19:09:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1759e-05 2021-01-15T19:09:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.193486+0000", 2021-01-15T19:09:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079234760000000005 2021-01-15T19:09:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.201409+0000", 2021-01-15T19:09:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0416e-05 2021-01-15T19:09:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:18.201460+0000", 2021-01-15T19:09:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0084513550000000007 2021-01-15T19:09:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.273 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.274 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:22.277 INFO:tasks.rados.rados.0.smithi001.stdout:366: done (3 left) 2021-01-15T19:09:22.277 INFO:tasks.rados.rados.0.smithi001.stdout:367: done (2 left) 2021-01-15T19:09:22.277 INFO:tasks.rados.rados.0.smithi001.stdout:368: done (1 left) 2021-01-15T19:09:22.277 INFO:tasks.rados.rados.0.smithi001.stdout:369: delete oid 1 current snap is 38 2021-01-15T19:09:22.278 INFO:tasks.rados.rados.0.smithi001.stdout:369: done (1 left) 2021-01-15T19:09:22.278 INFO:tasks.rados.rados.0.smithi001.stdout:370: read oid 11 snap 13 2021-01-15T19:09:22.278 INFO:tasks.rados.rados.0.smithi001.stdout:370: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:09:22.278 INFO:tasks.rados.rados.0.smithi001.stdout:371: read oid 40 snap 27 2021-01-15T19:09:22.279 INFO:tasks.rados.rados.0.smithi001.stdout:371: expect (ObjNum 106 snap 17 seq_num 106) 2021-01-15T19:09:22.279 INFO:tasks.rados.rados.0.smithi001.stdout:372: snap_create 2021-01-15T19:09:22.279 INFO:tasks.rados.rados.0.smithi001.stdout:370: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:09:22.279 INFO:tasks.rados.rados.0.smithi001.stdout:371: expect (ObjNum 106 snap 17 seq_num 106) 2021-01-15T19:09:22.279 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:09:22.280 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:22.275+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:22.280 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:22.377 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:22.375+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:22.415 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:22.415 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:22.416 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:22.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:22.510 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:22.509+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:22.578 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:22.576+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:22.598 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:22.599 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:22.609 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:22.679 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:22.678+0000 7ff28b671700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34825) UID: 0 2021-01-15T19:09:22.780 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:22.779+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:22.784 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:22.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.1s1 55/52 ECSubWrite(tid=100, reqid=osd.6.0:100, at_version=55'37, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:22.785 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.183697+0000", 2021-01-15T19:09:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.598104769000003, 2021-01-15T19:09:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017634312999999999, 2021-01-15T19:09:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183697+0000", 2021-01-15T19:09:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183697+0000", 2021-01-15T19:09:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6536999999999998e-05 2021-01-15T19:09:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183744+0000", 2021-01-15T19:09:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8940000000000002e-05 2021-01-15T19:09:22.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183763+0000", 2021-01-15T19:09:22.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4024e-05 2021-01-15T19:09:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183777+0000", 2021-01-15T19:09:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2080000000000006e-06 2021-01-15T19:09:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183786+0000", 2021-01-15T19:09:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4011000000000003e-05 2021-01-15T19:09:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183830+0000", 2021-01-15T19:09:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3817e-05 2021-01-15T19:09:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.183844+0000", 2021-01-15T19:09:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016034014999999999 2021-01-15T19:09:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.199878+0000", 2021-01-15T19:09:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001453761 2021-01-15T19:09:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.201332+0000", 2021-01-15T19:09:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017634312999999999 2021-01-15T19:09:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.800 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:22.801 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.346668264999998, 2021-01-15T19:09:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:09:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:09:22.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:09:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:09:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:09:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:09:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:09:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:09:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:09:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:09:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:09:22.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:09:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:09:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:09:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:09:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:09:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:09:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:09:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:09:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:09:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:09:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:09:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:09:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:09:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:09:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:09:22.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:09:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:09:22.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:09:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:09:22.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.823 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.346528964999997, 2021-01-15T19:09:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:09:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:09:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:09:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:09:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:09:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:09:22.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:09:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:09:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:09:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:09:22.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:09:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:09:22.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:09:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:09:22.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:09:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:09:22.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:09:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:09:22.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:09:22.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:09:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:09:22.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:09:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:09:22.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:09:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:09:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:09:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:09:22.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:09:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:09:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:09:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.346407859000003, 2021-01-15T19:09:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:09:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:09:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:09:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:09:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:09:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:09:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:09:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:09:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:09:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:09:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:09:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:09:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:09:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:09:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:09:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:09:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:09:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:09:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:09:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:09:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:09:22.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:09:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:09:22.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:09:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:09:22.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:09:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:09:22.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:09:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:09:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:09:22.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:09:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.576126985000002, 2021-01-15T19:09:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:09:22.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:09:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:09:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:09:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:09:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:09:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:09:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:09:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:09:22.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:09:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:09:22.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:09:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:09:22.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:09:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:09:22.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:09:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:09:22.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:09:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:09:22.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:09:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:09:22.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:09:22.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:09:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:09:22.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:09:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:09:22.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:09:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.567148807000002, 2021-01-15T19:09:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:09:22.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:09:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:09:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:09:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:09:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:09:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:09:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:09:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:09:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:09:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:09:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:09:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:09:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:09:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:09:22.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:09:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:09:22.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:09:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:09:22.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:09:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:09:22.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:09:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:09:22.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:09:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:09:22.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.900 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.566629106999997, 2021-01-15T19:09:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:09:22.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:09:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:09:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:09:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:09:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:09:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:09:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:09:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:09:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:09:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:09:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:09:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:09:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:09:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:09:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:09:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:09:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:09:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:09:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:09:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:09:22.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:09:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:09:22.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:09:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:09:22.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:09:22.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:09:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.564948153000003, 2021-01-15T19:09:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:09:22.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:09:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:09:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:09:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:09:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:09:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:09:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:09:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:09:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:09:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:09:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:09:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:09:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:09:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:09:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:09:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:09:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:09:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:09:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:09:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:09:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:09:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:09:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:09:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:09:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:09:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:22.936 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.564815199000002, 2021-01-15T19:09:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:09:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:22.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:09:22.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:22.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:22.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:09:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:09:22.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:09:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:09:22.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:09:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:09:22.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:09:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:09:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:09:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:09:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:09:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:09:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:09:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:09:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:09:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:09:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:09:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:09:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:09:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:09:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:09:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:09:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:09:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.952 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:22.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.602283928999999, 2021-01-15T19:09:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:09:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:09:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.955 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:09:22.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:09:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:09:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:09:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:09:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:09:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:09:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:09:22.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:09:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:09:22.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:09:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:09:22.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:09:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:09:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:09:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:09:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:09:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:22.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:09:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:09:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:22.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:09:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:09:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:09:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:09:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:09:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:09:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:22.971 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.602117335999999, 2021-01-15T19:09:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:09:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.972 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:09:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:09:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:09:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:09:22.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:09:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:09:22.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:09:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:09:22.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:09:22.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:09:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:09:22.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:09:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:09:22.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:09:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:09:22.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:09:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:09:22.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:09:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:09:22.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:09:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:09:22.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:09:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:09:22.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:09:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:09:22.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:09:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:09:22.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:09:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:22.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.403578542, 2021-01-15T19:09:22.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:09:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:22.991 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:09:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:09:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:09:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:09:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:09:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:09:22.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:09:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:09:22.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:09:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:09:22.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:09:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:09:22.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:09:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:09:23.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:09:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:09:23.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:09:23.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:09:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:09:23.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:09:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:09:23.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:09:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:09:23.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:09:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:09:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:09:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.401308497, 2021-01-15T19:09:23.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:09:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.009 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:09:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:09:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:09:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:09:23.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:09:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:09:23.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:09:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:09:23.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:09:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:09:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:09:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:09:23.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:09:23.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:09:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:09:23.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:09:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:09:23.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:09:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:09:23.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:09:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:09:23.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:09:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:09:23.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:09:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:09:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:09:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.396770306000001, 2021-01-15T19:09:23.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:09:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.027 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:09:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:09:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:09:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:09:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:09:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:09:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:09:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:09:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:09:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:09:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:09:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:09:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:09:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:09:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:09:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:09:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:09:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:09:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:09:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:09:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:09:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:09:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:09:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:09:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:09:23.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.043 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.394184434, 2021-01-15T19:09:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:09:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:09:23.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:23.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:23.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:09:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:09:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:09:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:09:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:09:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:09:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:09:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:09:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:09:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:09:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:09:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:23.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:09:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:09:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:09:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:09:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:09:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:09:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:09:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:09:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:09:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:09:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:09:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:09:23.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:09:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:09:23.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.061 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.393810099, 2021-01-15T19:09:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:09:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:09:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:23.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:23.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:09:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:09:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:09:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:09:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:09:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:09:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:09:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:09:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:09:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:09:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:09:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:09:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:09:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:09:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:09:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:09:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:09:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:09:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:09:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:09:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:09:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:09:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:09:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:09:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:09:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:23.079 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.393673604, 2021-01-15T19:09:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:09:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.080 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:09:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.081 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:23.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:23.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:09:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:09:23.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:09:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:09:23.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:09:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:09:23.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:09:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:09:23.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:09:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:09:23.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:09:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:09:23.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:09:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:09:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:09:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:23.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:09:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:09:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:09:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:09:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:09:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:09:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:09:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:09:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.16s0 85/84 ECSubWriteReply(tid=160, last_complete=26'11, committed=1, applied=1))", 2021-01-15T19:09:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.347698+0000", 2021-01-15T19:09:23.096 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.4341046980000005, 2021-01-15T19:09:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019258414000000001, 2021-01-15T19:09:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347698+0000", 2021-01-15T19:09:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347698+0000", 2021-01-15T19:09:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6031e-05 2021-01-15T19:09:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347744+0000", 2021-01-15T19:09:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0329000000000001e-05 2021-01-15T19:09:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347754+0000", 2021-01-15T19:09:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7512000000000001e-05 2021-01-15T19:09:23.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347771+0000", 2021-01-15T19:09:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8899999999999996e-06 2021-01-15T19:09:23.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347780+0000", 2021-01-15T19:09:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016302565000000001 2021-01-15T19:09:23.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364083+0000", 2021-01-15T19:09:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028730869999999999 2021-01-15T19:09:23.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366956+0000", 2021-01-15T19:09:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019258414000000001 2021-01-15T19:09:23.105 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.106 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:889 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [copy-from ver 38] snapc 3a=[3a,38,36,2f,2a,26,25,1b,19] ondisk+write+known_if_redirected e86)", 2021-01-15T19:09:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.4113855770000008, 2021-01-15T19:09:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999, 2021-01-15T19:09:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 889 2021-01-15T19:09:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3864e-05 2021-01-15T19:09:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370451+0000", 2021-01-15T19:09:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0156999999999999e-05 2021-01-15T19:09:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370461+0000", 2021-01-15T19:09:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2923e-05 2021-01-15T19:09:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370474+0000", 2021-01-15T19:09:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0399999999999998e-06 2021-01-15T19:09:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370483+0000", 2021-01-15T19:09:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9975999999999994e-05 2021-01-15T19:09:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370573+0000", 2021-01-15T19:09:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001672 2021-01-15T19:09:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370740+0000", 2021-01-15T19:09:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022655103999999999 2021-01-15T19:09:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.393395+0000", 2021-01-15T19:09:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087490950000000001 2021-01-15T19:09:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.402144+0000", 2021-01-15T19:09:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020087099999999999 2021-01-15T19:09:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404153+0000", 2021-01-15T19:09:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0517000000000001e-05 2021-01-15T19:09:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404243+0000", 2021-01-15T19:09:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999 2021-01-15T19:09:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:09:23.122 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.4112758840000001, 2021-01-15T19:09:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:09:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:09:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.124 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:09:23.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:09:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:09:23.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:09:23.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:09:23.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:09:23.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:09:23.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:09:23.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:09:23.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:09:23.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:09:23.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:23.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:09:23.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:09:23.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:09:23.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:09:23.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:23.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:09:23.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:09:23.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:09:23.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:09:23.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:09:23.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:09:23.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:09:23.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:09:23.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.139 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:23.140 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:09:23.140 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:23.140 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2021-01-15T19:09:23.141 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2021-01-15T19:09:23.141 INFO:teuthology.orchestra.run:waiting for 300 2021-01-15T19:09:23.142 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:23.141+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:23.143 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:23.176 INFO:tasks.ceph.osd.6:Stopped 2021-01-15T19:09:23.177 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2021-01-15T19:09:23.177 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi136.front.sepia.ceph.com 2021-01-15T19:09:23.177 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2021-01-15T19:09:23.205 INFO:tasks.rados.rados.0.smithi001.stdout:370: done (3 left) 2021-01-15T19:09:23.205 INFO:tasks.rados.rados.0.smithi001.stdout:371: done (2 left) 2021-01-15T19:09:23.205 INFO:tasks.rados.rados.0.smithi001.stdout:372: done (1 left) 2021-01-15T19:09:23.205 INFO:tasks.rados.rados.0.smithi001.stdout:373: rollback oid 9 current snap is 39 2021-01-15T19:09:23.206 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 9 to 36 2021-01-15T19:09:23.206 INFO:tasks.rados.rados.0.smithi001.stdout:374: read oid 47 snap -1 2021-01-15T19:09:23.206 INFO:tasks.rados.rados.0.smithi001.stdout:374: expect (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:09:23.206 INFO:tasks.rados.rados.0.smithi001.stdout:375: rollback oid 46 current snap is 39 2021-01-15T19:09:23.206 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 46 to 25 2021-01-15T19:09:23.207 INFO:tasks.rados.rados.0.smithi001.stdout:376: snap_remove snap 14 2021-01-15T19:09:23.233 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2021-01-15T19:09:23.233 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2021-01-15T19:09:23.243 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:23.242+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:23.338 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:23.338 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:23.339 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:23.339 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:23.344 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:23.343+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:23.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:23.445 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:23.444+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:23.521 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:23.521 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:23.522 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:23.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:23.546 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:23.545+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:23.665 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:23.665 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:23.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:23.666 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:23.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.666 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:23.667 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:23.667 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.414233347, 2021-01-15T19:09:23.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:23.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:23.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.669 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:23.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:23.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:23.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:23.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:23.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:23.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:23.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:23.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:23.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:23.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:23.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:23.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:23.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:23.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:23.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:23.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:23.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:23.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:23.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:23.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:23.682 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.682 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.682 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.683 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:23.684 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:23.684 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.414042403000003, 2021-01-15T19:09:23.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:23.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:23.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.687 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:23.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:23.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:23.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:23.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:23.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:23.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:23.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:23.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:23.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:23.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:23.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:23.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:23.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:23.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:23.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:23.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:23.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:23.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:23.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:23.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:23.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:23.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:23.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:23.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:23.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:23.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:23.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.703 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.703 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:23.703 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.475090305999998, 2021-01-15T19:09:23.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:23.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.704 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.705 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:23.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.705 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:23.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:23.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:23.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:23.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:23.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:23.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:23.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:23.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:23.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:23.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:23.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:23.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:23.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:23.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:23.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:23.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:23.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:23.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:23.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:23.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:23.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:23.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:23.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.721 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.722 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:23.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:23.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.472369845000003, 2021-01-15T19:09:23.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:23.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.723 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.723 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.724 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:23.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.724 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:23.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:23.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:23.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:23.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:23.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:23.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:23.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:23.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:23.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:23.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:23.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:23.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:23.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:23.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:23.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:23.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:23.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:23.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:23.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:23.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:23.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:23.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:23.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:23.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:23.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:23.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:23.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.739 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.739 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.740 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:23.740 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.472257266, 2021-01-15T19:09:23.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:23.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.741 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.742 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:23.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.742 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:23.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:23.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:23.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:23.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:23.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:23.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:23.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:23.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:23.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:23.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:23.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:23.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:23.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:23.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:23.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:23.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:23.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:23.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:23.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:23.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:23.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:23.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.756 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.757 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:23.757 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:23.757 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.472127362000002, 2021-01-15T19:09:23.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:23.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:23.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.759 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:23.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:23.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:23.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:23.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:23.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:23.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:23.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:23.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:23.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:23.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:23.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:23.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:23.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:23.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.448922093, 2021-01-15T19:09:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:23.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:23.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:23.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:23.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:23.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:23.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:23.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:23.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:23.787 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.448803894999998, 2021-01-15T19:09:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:23.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:23.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:23.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:23.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:23.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:23.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:23.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:23.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:23.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.448691205000003, 2021-01-15T19:09:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:23.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:23.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:23.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:23.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:23.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:23.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:23.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:23.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:23.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.443089454999999, 2021-01-15T19:09:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:23.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.434620731000003, 2021-01-15T19:09:23.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:23.845 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.430102148000003, 2021-01-15T19:09:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:23.860 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.425695050999998, 2021-01-15T19:09:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.874 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:23.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.425212924999997, 2021-01-15T19:09:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.877 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:23.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:23.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:23.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:23.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:23.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:23.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:23.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:23.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:23.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:23.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:23.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.890 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.425084378999998, 2021-01-15T19:09:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:23.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:23.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:23.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:23.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:23.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:23.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:23.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:23.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:23.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:23.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:23.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:23.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:23.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.904 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.416635239000001, 2021-01-15T19:09:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:23.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:23.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:23.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:23.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:23.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:23.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:23.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:23.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.457960301, 2021-01-15T19:09:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:23.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:23.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.333775619000001, 2021-01-15T19:09:23.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.472414331, 2021-01-15T19:09:23.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:23.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.480773653, 2021-01-15T19:09:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:23.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.958 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:23.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:23.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:23.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:23.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:23.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.972 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:23.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:23.973 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:23.974 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:23.975 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:23.645+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:23.976 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:23.976 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:23.973+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:24.097 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:24.075+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:24.104 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:24.104 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:24.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.105 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3.e3171838 (undecoded) ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:24.105 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:24.105 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.89790773199999996, 2021-01-15T19:09:24.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.897930534, 2021-01-15T19:09:24.105 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:24.106 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "delayed", 2021-01-15T19:09:24.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:24.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:24.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:24.107 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:24.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.107 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:24.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:24.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:24.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:24.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:24.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:24.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:24.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:24.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:24.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:24.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:24.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:24.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:24.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:24.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:24.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:24.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:24.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:24.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:24.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:24.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:24.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:24.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:24.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:24.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:24.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:24.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:24.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:24.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:24.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:24.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053079269999999996 2021-01-15T19:09:24.117 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:24.118 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:24.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:24.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.118 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3.e3171838 (undecoded) ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:24.119 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:24.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.89781908300000002, 2021-01-15T19:09:24.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.89789551999999995, 2021-01-15T19:09:24.119 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:24.119 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "delayed", 2021-01-15T19:09:24.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:24.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:24.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:24.120 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:24.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.121 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:24.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:24.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:24.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:24.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:24.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:24.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:24.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:24.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:24.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:24.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:24.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:24.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:24.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:24.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:24.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:24.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:24.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:24.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:24.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:24.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:24.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:24.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:24.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:24.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:24.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:24.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:24.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:24.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:24.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:24.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:24.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:24.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052603720000000001 2021-01-15T19:09:24.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:24.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:24.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:24.132 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:24.132 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2021-01-15T19:09:24.132 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 2 2021-01-15T19:09:24.132 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:24.133 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:24.177 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:24.178 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:24.177+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:24.197 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:24.213 INFO:tasks.rados.rados.0.smithi001.stdout:376: done (4 left) 2021-01-15T19:09:24.214 INFO:tasks.rados.rados.0.smithi001.stdout:377: copy_from oid 20 from oid 8 current snap is 39 2021-01-15T19:09:24.219 INFO:tasks.rados.rados.0.smithi001.stdout:378: read oid 30 snap 33 2021-01-15T19:09:24.219 INFO:tasks.rados.rados.0.smithi001.stdout:378: expect deleted 2021-01-15T19:09:24.220 INFO:tasks.rados.rados.0.smithi001.stdout:379: read oid 44 snap -1 2021-01-15T19:09:24.220 INFO:tasks.rados.rados.0.smithi001.stdout:379: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:24.221 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 16 current snap is 39 2021-01-15T19:09:24.221 INFO:tasks.rados.rados.0.smithi001.stdout:380: seq_num 148 ranges {0=81920} 2021-01-15T19:09:24.221 INFO:tasks.rados.rados.0.smithi001.stdout:380: writing smithi00140323-16 from 0 to 81920 tid 1 2021-01-15T19:09:24.222 INFO:tasks.rados.rados.0.smithi001.stdout:381: read oid 34 snap -1 2021-01-15T19:09:24.222 INFO:tasks.rados.rados.0.smithi001.stdout:381: expect (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:09:24.222 INFO:tasks.rados.rados.0.smithi001.stdout:382: read oid 35 snap -1 2021-01-15T19:09:24.223 INFO:tasks.rados.rados.0.smithi001.stdout:382: expect deleted 2021-01-15T19:09:24.223 INFO:tasks.rados.rados.0.smithi001.stdout:383: rollback oid 25 current snap is 39 2021-01-15T19:09:24.224 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 25 to 34 2021-01-15T19:09:24.224 INFO:tasks.rados.rados.0.smithi001.stdout:384: rollback oid 36 current snap is 39 2021-01-15T19:09:24.224 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 36 to 32 2021-01-15T19:09:24.224 INFO:tasks.rados.rados.0.smithi001.stdout:385: snap_create 2021-01-15T19:09:24.225 INFO:tasks.rados.rados.0.smithi001.stdout:373: finishing rollback tid 0 to smithi00140323-9 2021-01-15T19:09:24.225 INFO:tasks.rados.rados.0.smithi001.stdout:373: finishing rollback tid 1 to smithi00140323-9 2021-01-15T19:09:24.225 INFO:tasks.rados.rados.0.smithi001.stdout:373: finishing rollback tid 2 to smithi00140323-9 2021-01-15T19:09:24.225 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 9 v 50 (ObjNum 139 snap 34 seq_num 139) dirty exists 2021-01-15T19:09:24.225 INFO:tasks.rados.rados.0.smithi001.stdout:374: expect (ObjNum 86 snap 15 seq_num 86) 2021-01-15T19:09:24.238 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:24.270 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:24.278 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:24.277+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:24.278 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:24.278 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:24.279 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:24.282 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:09:24.288 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:24.290 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:24.313 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:24.334 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:24.334 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.335 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.335 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.335 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.335 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.336 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.336 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:24.336 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:24.336 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:24.337 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:24.338 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:24.360 INFO:tasks.rados.rados.0.smithi001.stdout:375: finishing rollback tid 1 to smithi00140323-46 2021-01-15T19:09:24.360 INFO:tasks.rados.rados.0.smithi001.stdout:375: finishing rollback tid 2 to smithi00140323-46 2021-01-15T19:09:24.360 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 46 v 58 (ObjNum 45 snap 0 seq_num 45) dirty exists 2021-01-15T19:09:24.361 INFO:tasks.rados.rados.0.smithi001.stdout:383: finishing rollback tid 0 to smithi00140323-25 2021-01-15T19:09:24.361 INFO:tasks.rados.rados.0.smithi001.stdout:380: finishing write tid 1 to smithi00140323-16 2021-01-15T19:09:24.361 INFO:tasks.rados.rados.0.smithi001.stdout:383: finishing rollback tid 1 to smithi00140323-25 2021-01-15T19:09:24.361 INFO:tasks.rados.rados.0.smithi001.stdout:383: finishing rollback tid 2 to smithi00140323-25 2021-01-15T19:09:24.361 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 129 (ObjNum 24 snap 0 seq_num 24) dirty exists 2021-01-15T19:09:24.362 INFO:tasks.rados.rados.0.smithi001.stdout:380: finishing write tid 2 to smithi00140323-16 2021-01-15T19:09:24.362 INFO:tasks.rados.rados.0.smithi001.stdout:380: finishing write tid 3 to smithi00140323-16 2021-01-15T19:09:24.362 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 95 (ObjNum 148 snap 39 seq_num 148) dirty exists 2021-01-15T19:09:24.362 INFO:tasks.rados.rados.0.smithi001.stdout:380: left oid 16 (ObjNum 148 snap 39 seq_num 148) 2021-01-15T19:09:24.363 INFO:tasks.rados.rados.0.smithi001.stdout:384: finishing rollback tid 0 to smithi00140323-36 2021-01-15T19:09:24.363 INFO:tasks.rados.rados.0.smithi001.stdout:384: finishing rollback tid 1 to smithi00140323-36 2021-01-15T19:09:24.363 INFO:tasks.rados.rados.0.smithi001.stdout:381: expect (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:09:24.378 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:24.377+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:24.421 INFO:teuthology.orchestra.run.smithi136.stdout:3.bs1 2021-01-15T19:09:24.421 INFO:teuthology.orchestra.run.smithi136.stdout:3.5s1 2021-01-15T19:09:24.421 INFO:teuthology.orchestra.run.smithi136.stdout:3.4s0 2021-01-15T19:09:24.422 INFO:teuthology.orchestra.run.smithi136.stdout:3.1s0 2021-01-15T19:09:24.422 INFO:teuthology.orchestra.run.smithi136.stdout:3.18s1 2021-01-15T19:09:24.422 INFO:teuthology.orchestra.run.smithi136.stdout:3.15s1 2021-01-15T19:09:24.422 INFO:teuthology.orchestra.run.smithi136.stdout:3.14s2 2021-01-15T19:09:24.422 INFO:teuthology.orchestra.run.smithi136.stdout:3.12s0 2021-01-15T19:09:24.423 INFO:teuthology.orchestra.run.smithi136.stdout:2.c 2021-01-15T19:09:24.423 INFO:teuthology.orchestra.run.smithi136.stdout:2.a 2021-01-15T19:09:24.423 INFO:teuthology.orchestra.run.smithi136.stdout:2.7 2021-01-15T19:09:24.423 INFO:teuthology.orchestra.run.smithi136.stdout:2.4 2021-01-15T19:09:24.424 INFO:teuthology.orchestra.run.smithi136.stdout:2.2 2021-01-15T19:09:24.424 INFO:teuthology.orchestra.run.smithi136.stdout:2.0 2021-01-15T19:09:24.424 INFO:teuthology.orchestra.run.smithi136.stdout:1.a 2021-01-15T19:09:24.424 INFO:teuthology.orchestra.run.smithi136.stdout:1.7 2021-01-15T19:09:24.424 INFO:teuthology.orchestra.run.smithi136.stdout:3.7s1 2021-01-15T19:09:24.425 INFO:teuthology.orchestra.run.smithi136.stdout:1.6 2021-01-15T19:09:24.425 INFO:teuthology.orchestra.run.smithi136.stdout:3.cs2 2021-01-15T19:09:24.425 INFO:teuthology.orchestra.run.smithi136.stdout:3.11s0 2021-01-15T19:09:24.425 INFO:teuthology.orchestra.run.smithi136.stdout:1.5 2021-01-15T19:09:24.426 INFO:teuthology.orchestra.run.smithi136.stdout:1.1 2021-01-15T19:09:24.459 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:24.459 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:24.468 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:24.512 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:24.511+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:24.567 INFO:tasks.rados.rados.0.smithi001.stdout:384: finishing rollback tid 2 to smithi00140323-36 2021-01-15T19:09:24.568 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 79 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:24.568 INFO:tasks.rados.rados.0.smithi001.stdout:379: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:24.579 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:24.578+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:24.594 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:24.594 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:24.604 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:24.688 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:24.687+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:24.774 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:24.775 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:24.782 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:24.781+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:24.784 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:24.883 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:24.882+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:24.941 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 3.cs2 --file /var/log/ceph/exp.3.cs2.6 2021-01-15T19:09:24.950 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:24.950 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:24.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:25.002 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:25.001+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:25.084 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:25.083+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:25.126 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:25.127 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:25.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:25.185 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:25.184+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:25.205 INFO:tasks.rados.rados.0.smithi001.stdout:373: done (12 left) 2021-01-15T19:09:25.206 INFO:tasks.rados.rados.0.smithi001.stdout:374: done (11 left) 2021-01-15T19:09:25.206 INFO:tasks.rados.rados.0.smithi001.stdout:375: done (10 left) 2021-01-15T19:09:25.206 INFO:tasks.rados.rados.0.smithi001.stdout:378: done (9 left) 2021-01-15T19:09:25.206 INFO:tasks.rados.rados.0.smithi001.stdout:379: done (8 left) 2021-01-15T19:09:25.207 INFO:tasks.rados.rados.0.smithi001.stdout:380: done (7 left) 2021-01-15T19:09:25.207 INFO:tasks.rados.rados.0.smithi001.stdout:381: done (6 left) 2021-01-15T19:09:25.207 INFO:tasks.rados.rados.0.smithi001.stdout:382: done (5 left) 2021-01-15T19:09:25.207 INFO:tasks.rados.rados.0.smithi001.stdout:383: done (4 left) 2021-01-15T19:09:25.207 INFO:tasks.rados.rados.0.smithi001.stdout:384: done (3 left) 2021-01-15T19:09:25.208 INFO:tasks.rados.rados.0.smithi001.stdout:385: done (2 left) 2021-01-15T19:09:25.208 INFO:tasks.rados.rados.0.smithi001.stdout:386: delete oid 47 current snap is 40 2021-01-15T19:09:25.216 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:09:25.216 INFO:tasks.rados.rados.0.smithi001.stdout:386: done (2 left) 2021-01-15T19:09:25.217 INFO:tasks.rados.rados.0.smithi001.stdout:387: delete oid 49 current snap is 40 2021-01-15T19:09:25.222 INFO:tasks.rados.rados.0.smithi001.stdout:387: done (2 left) 2021-01-15T19:09:25.222 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 30 current snap is 40 2021-01-15T19:09:25.222 INFO:tasks.rados.rados.0.smithi001.stdout:388: seq_num 151 ranges {0=131072} 2021-01-15T19:09:25.223 INFO:tasks.rados.rados.0.smithi001.stdout:388: writing smithi00140323-30 from 0 to 131072 tid 1 2021-01-15T19:09:25.223 INFO:tasks.rados.rados.0.smithi001.stdout:389: delete oid 17 current snap is 40 2021-01-15T19:09:25.225 INFO:tasks.rados.rados.0.smithi001.stdout:389: done (3 left) 2021-01-15T19:09:25.226 INFO:tasks.rados.rados.0.smithi001.stdout:390: delete oid 43 current snap is 40 2021-01-15T19:09:25.226 INFO:tasks.rados.rados.0.smithi001.stdout:388: finishing write tid 1 to smithi00140323-30 2021-01-15T19:09:25.227 INFO:tasks.rados.rados.0.smithi001.stdout:388: finishing write tid 2 to smithi00140323-30 2021-01-15T19:09:25.229 INFO:tasks.rados.rados.0.smithi001.stdout:388: finishing write tid 3 to smithi00140323-30 2021-01-15T19:09:25.229 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 30 v 41 (ObjNum 151 snap 40 seq_num 151) dirty exists 2021-01-15T19:09:25.229 INFO:tasks.rados.rados.0.smithi001.stdout:388: left oid 30 (ObjNum 151 snap 40 seq_num 151) 2021-01-15T19:09:25.229 INFO:tasks.rados.rados.0.smithi001.stdout:388: done (3 left) 2021-01-15T19:09:25.230 INFO:tasks.rados.rados.0.smithi001.stdout:390: done (2 left) 2021-01-15T19:09:25.230 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 41 current snap is 40 2021-01-15T19:09:25.230 INFO:tasks.rados.rados.0.smithi001.stdout:391: seq_num 154 ranges {2465792=180224} 2021-01-15T19:09:25.244 INFO:tasks.rados.rados.0.smithi001.stdout:391: writing smithi00140323-41 from 2465792 to 2646016 tid 1 2021-01-15T19:09:25.245 INFO:tasks.rados.rados.0.smithi001.stdout:392: read oid 17 snap -1 2021-01-15T19:09:25.245 INFO:tasks.rados.rados.0.smithi001.stdout:392: expect deleted 2021-01-15T19:09:25.245 INFO:tasks.rados.rados.0.smithi001.stdout:393: read oid 30 snap -1 2021-01-15T19:09:25.245 INFO:tasks.rados.rados.0.smithi001.stdout:393: expect (ObjNum 151 snap 40 seq_num 151) 2021-01-15T19:09:25.246 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 28 current snap is 40 2021-01-15T19:09:25.246 INFO:tasks.rados.rados.0.smithi001.stdout:394: seq_num 155 ranges {1810432=204800} 2021-01-15T19:09:25.256 INFO:tasks.rados.rados.0.smithi001.stdout:394: writing smithi00140323-28 from 1810432 to 2015232 tid 1 2021-01-15T19:09:25.256 INFO:tasks.rados.rados.0.smithi001.stdout:395: snap_remove snap 37 2021-01-15T19:09:25.285 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:25.284+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:25.324 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:25.325 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:25.337 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:25.387 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:25.385+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:25.463 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:25.464 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:25.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:25.517 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:25.516+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:25.589 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:25.587+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:25.611 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:25.611 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:25.621 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:25.690 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:25.689+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:25.793 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:25.791+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:25.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:25.894 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:25.892+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:25.934 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:25.935 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:25.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:26.028 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:26.027+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:26.057 INFO:teuthology.orchestra.run.smithi136.stderr:Exporting 3.cs2 info 3.cs2( v 91'152 (0'0,91'152] local-lis/les=86/87 n=16 ec=21/21 lis/c=86/86 les/c/f=87/87/0 sis=86) 2021-01-15T19:09:26.062 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:30718dd9:::smithi00140323-34:36# 2021-01-15T19:09:26.065 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:30718dd9:::smithi00140323-34:head# 2021-01-15T19:09:26.072 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:30ee460e:::smithi00140323-45:1c# 2021-01-15T19:09:26.073 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:30ee460e:::smithi00140323-45:36# 2021-01-15T19:09:26.074 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:30ee460e:::smithi00140323-45:head# 2021-01-15T19:09:26.078 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3e4f5332:::smithi00140323-33:1b# 2021-01-15T19:09:26.079 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3e4f5332:::smithi00140323-33:28# 2021-01-15T19:09:26.083 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3e4f5332:::smithi00140323-33:2d# 2021-01-15T19:09:26.087 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3e4f5332:::smithi00140323-33:32# 2021-01-15T19:09:26.090 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3e4f5332:::smithi00140323-33:3a# 2021-01-15T19:09:26.092 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3e4f5332:::smithi00140323-33:head# 2021-01-15T19:09:26.095 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:26.093+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:26.098 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3ef45b66:::smithi00140323-11:1c# 2021-01-15T19:09:26.099 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3ef45b66:::smithi00140323-11:head# 2021-01-15T19:09:26.102 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3f312ec7:::smithi00140323-7:1c# 2021-01-15T19:09:26.103 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3f312ec7:::smithi00140323-7:3a# 2021-01-15T19:09:26.105 INFO:teuthology.orchestra.run.smithi136.stderr:Read 2#3:3f312ec7:::smithi00140323-7:head# 2021-01-15T19:09:26.112 INFO:teuthology.orchestra.run.smithi136.stderr:Export successful 2021-01-15T19:09:26.116 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:26.117 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:26.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:26.208 INFO:tasks.rados.rados.0.smithi001.stdout:395: done (6 left) 2021-01-15T19:09:26.208 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 38 current snap is 40 2021-01-15T19:09:26.209 INFO:tasks.rados.rados.0.smithi001.stdout:396: seq_num 156 ranges {4456448=221184} 2021-01-15T19:09:26.209 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:26.207+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:26.237 INFO:tasks.rados.rados.0.smithi001.stdout:396: writing smithi00140323-38 from 4456448 to 4677632 tid 1 2021-01-15T19:09:26.238 INFO:tasks.rados.rados.0.smithi001.stdout:397: read oid 48 snap -1 2021-01-15T19:09:26.238 INFO:tasks.rados.rados.0.smithi001.stdout:397: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:26.238 INFO:tasks.rados.rados.0.smithi001.stdout:398: snap_remove snap 39 2021-01-15T19:09:26.297 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:26.295+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:26.309 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:26.310 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:26.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:26.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:26.403 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:26.402+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:26.495 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:26.495 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:26.499 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:26.499+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:26.505 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:26.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:26.529 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:26.550 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:26.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:26.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:26.601 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:26.600+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:26.612 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:26.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 3.cs2 2021-01-15T19:09:26.634 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:26.634 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.634 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.635 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.635 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.635 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.635 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.635 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:26.636 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:26.636 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:26.637 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:26.637 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:26.643 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:26.644 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:26.654 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:26.716 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:26.715+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:26.803 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:26.803+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:26.827 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:26.828 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:26.837 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:26.904 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:26.904+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:26.971 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.53313722, 2021-01-15T19:09:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:09:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:09:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:09:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:09:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:09:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:09:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:09:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:09:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:09:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:09:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:09:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:09:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:09:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:09:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:09:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:09:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:09:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:09:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:09:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:09:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:09:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:09:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:09:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:09:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:09:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:09:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:09:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:09:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:09:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:09:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.53299792, 2021-01-15T19:09:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:09:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:09:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:09:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:09:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:09:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:09:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:09:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:09:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:09:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:09:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:09:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:09:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:09:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:09:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:09:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:09:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:09:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:09:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:09:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:09:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:09:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:09:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:09:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:09:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:09:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:09:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:09:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:09:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:09:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:09:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.532876813999998, 2021-01-15T19:09:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:09:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:09:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:09:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:09:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:09:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:09:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:09:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:09:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:09:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:09:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:09:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:09:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:09:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:09:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:09:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:09:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:09:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:09:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:09:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:09:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:09:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:09:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:09:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:09:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:09:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:09:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:09:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:09:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:09:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:09:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:09:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.762595939999997, 2021-01-15T19:09:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:09:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:09:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:09:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:09:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:09:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:09:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:09:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:09:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:09:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:09:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:09:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:09:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:09:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:09:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:09:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:09:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:09:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:09:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:09:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:09:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:09:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:09:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:09:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:09:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:09:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:09:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:09:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.753617761999998, 2021-01-15T19:09:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:09:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:09:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:09:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:09:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:09:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:09:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:09:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:09:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:09:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:09:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:09:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:09:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:09:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:09:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:09:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:09:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:09:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:09:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:09:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:09:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:09:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:09:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:09:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:09:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:09:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.753098061999999, 2021-01-15T19:09:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:09:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:09:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:09:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:09:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:09:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:09:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:09:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:09:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:09:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:09:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:09:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:09:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:09:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:09:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:09:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:09:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:09:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:09:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:09:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:09:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:09:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:09:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:09:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:09:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:09:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:09:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:09:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.751417107999998, 2021-01-15T19:09:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:09:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:09:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:09:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:09:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:09:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:09:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:09:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:09:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:09:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:09:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:09:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:09:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:09:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:09:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:09:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:09:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:09:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:09:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:09:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:09:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:09:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:09:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:09:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:09:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:09:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:09:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.751284153999997, 2021-01-15T19:09:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:09:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:09:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:09:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:09:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:09:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:09:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:09:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:09:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:09:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:09:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:09:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:09:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:09:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:09:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:09:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:09:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:09:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:09:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:09:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:09:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:09:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:09:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:09:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:09:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:09:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.788752883999997, 2021-01-15T19:09:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:09:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:09:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:09:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:09:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:09:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:09:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:09:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:09:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:09:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:09:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:09:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:09:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:09:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:09:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:09:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:09:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:09:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:09:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:09:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:09:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:09:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:09:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:09:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:09:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:09:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:09:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:09:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.788586291000001, 2021-01-15T19:09:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:09:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:09:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:09:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:09:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:09:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:09:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:09:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:09:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:09:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:09:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:09:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:09:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:09:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:09:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:09:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:09:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:09:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:09:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:09:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:09:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:09:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:09:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:09:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:09:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:09:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:09:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:09:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:09:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:09:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.590047497, 2021-01-15T19:09:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:09:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:09:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:09:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:09:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:09:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:09:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:09:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:09:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:09:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:09:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:09:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:09:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:09:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:09:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:09:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:09:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:09:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:09:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:09:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:09:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:09:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:09:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:09:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:09:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:09:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:09:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:09:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.587777451999999, 2021-01-15T19:09:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:09:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:09:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:09:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:09:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:09:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:09:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:09:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:09:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:09:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:09:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:09:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:09:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:09:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:09:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:09:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:09:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:09:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:09:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:09:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:09:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:09:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:09:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:09:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:09:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:09:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:09:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:09:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.583239260999999, 2021-01-15T19:09:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:09:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:09:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:09:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:09:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:09:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:09:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:09:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:09:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:09:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:09:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:09:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:09:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:09:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:09:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:09:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:09:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:09:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:09:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:09:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:09:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:09:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:09:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:09:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:09:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:09:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:09:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.580653389, 2021-01-15T19:09:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:09:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:09:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:09:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:09:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:09:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:09:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:09:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:09:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:09:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:09:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:09:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:09:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:09:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:09:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:09:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:09:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:09:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:09:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:09:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:09:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:09:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:09:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:09:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:09:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:09:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:09:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:09:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.580279054, 2021-01-15T19:09:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:09:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:09:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:09:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:09:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:09:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:09:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:09:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:09:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:09:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:09:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:09:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:09:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:09:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:09:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:09:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:09:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:09:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:09:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:09:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:09:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:09:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:09:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:09:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:09:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:09:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:09:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:09:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.580142559, 2021-01-15T19:09:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:09:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:09:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:09:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:09:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:09:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:09:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:09:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:09:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:09:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:09:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:09:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:09:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:09:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:09:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:09:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:09:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:09:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:09:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:09:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:09:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:09:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:09:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:09:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:09:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:09:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:889 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [copy-from ver 38] snapc 3a=[3a,38,36,2f,2a,26,25,1b,19] ondisk+write+known_if_redirected e86)", 2021-01-15T19:09:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 12.597854531999999, 2021-01-15T19:09:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999, 2021-01-15T19:09:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 889 2021-01-15T19:09:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3864e-05 2021-01-15T19:09:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370451+0000", 2021-01-15T19:09:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0156999999999999e-05 2021-01-15T19:09:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370461+0000", 2021-01-15T19:09:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2923e-05 2021-01-15T19:09:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370474+0000", 2021-01-15T19:09:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0399999999999998e-06 2021-01-15T19:09:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370483+0000", 2021-01-15T19:09:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9975999999999994e-05 2021-01-15T19:09:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370573+0000", 2021-01-15T19:09:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001672 2021-01-15T19:09:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370740+0000", 2021-01-15T19:09:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022655103999999999 2021-01-15T19:09:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.393395+0000", 2021-01-15T19:09:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087490950000000001 2021-01-15T19:09:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.402144+0000", 2021-01-15T19:09:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020087099999999999 2021-01-15T19:09:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404153+0000", 2021-01-15T19:09:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0517000000000001e-05 2021-01-15T19:09:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404243+0000", 2021-01-15T19:09:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999 2021-01-15T19:09:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:09:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 12.597744839000001, 2021-01-15T19:09:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:09:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:09:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:09:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:09:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:09:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:09:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:09:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:09:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:09:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:09:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:09:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:09:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:09:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:09:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:09:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:09:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:09:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:09:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:09:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:09:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:09:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:09:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:09:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:09:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:09:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:968 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.206245+0000", 2021-01-15T19:09:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.7620263170000001, 2021-01-15T19:09:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019846322, 2021-01-15T19:09:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 968 2021-01-15T19:09:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.206245+0000", 2021-01-15T19:09:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.206245+0000", 2021-01-15T19:09:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6746000000000003e-05 2021-01-15T19:09:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.206292+0000", 2021-01-15T19:09:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2928999999999999e-05 2021-01-15T19:09:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.206305+0000", 2021-01-15T19:09:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0990000000000001e-05 2021-01-15T19:09:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.206326+0000", 2021-01-15T19:09:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7769999999999997e-06 2021-01-15T19:09:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.206335+0000", 2021-01-15T19:09:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024468290000000002 2021-01-15T19:09:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208782+0000", 2021-01-15T19:09:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000150481 2021-01-15T19:09:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208933+0000", 2021-01-15T19:09:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00088763399999999999 2021-01-15T19:09:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209820+0000", 2021-01-15T19:09:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015956000000000001 2021-01-15T19:09:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.225776+0000", 2021-01-15T19:09:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000200986 2021-01-15T19:09:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.225977+0000", 2021-01-15T19:09:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011395000000000001 2021-01-15T19:09:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.226091+0000", 2021-01-15T19:09:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019846322 2021-01-15T19:09:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:970 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.208053+0000", 2021-01-15T19:09:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.760217811, 2021-01-15T19:09:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019367307, 2021-01-15T19:09:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 970 2021-01-15T19:09:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208053+0000", 2021-01-15T19:09:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208053+0000", 2021-01-15T19:09:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2543000000000003e-05 2021-01-15T19:09:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208086+0000", 2021-01-15T19:09:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0230000000000004e-06 2021-01-15T19:09:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208093+0000", 2021-01-15T19:09:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1575e-05 2021-01-15T19:09:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208105+0000", 2021-01-15T19:09:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3759999999999999e-06 2021-01-15T19:09:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.208110+0000", 2021-01-15T19:09:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016662289 2021-01-15T19:09:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.224772+0000", 2021-01-15T19:09:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000157093 2021-01-15T19:09:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.224929+0000", 2021-01-15T19:09:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00054811400000000004 2021-01-15T19:09:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.225477+0000", 2021-01-15T19:09:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00088478199999999997 2021-01-15T19:09:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.226362+0000", 2021-01-15T19:09:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010020420000000001 2021-01-15T19:09:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227364+0000", 2021-01-15T19:09:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.647e-05 2021-01-15T19:09:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227421+0000", 2021-01-15T19:09:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019367307 2021-01-15T19:09:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:27.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:09:27.414 INFO:tasks.rados.rados.0.smithi001.stdout:398: done (8 left) 2021-01-15T19:09:27.415 INFO:tasks.rados.rados.0.smithi001.stdout:399: snap_remove snap 34 2021-01-15T19:09:27.415 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:27.413+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:27.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:27.514 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:27.513+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:27.597 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:27.597 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:27.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:27.648 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:27.647+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:27.716 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:27.714+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:27.770 INFO:teuthology.orchestra.run.smithi136.stdout: marking collection for removal 2021-01-15T19:09:27.771 INFO:teuthology.orchestra.run.smithi136.stdout:setting '_remove' omap key 2021-01-15T19:09:27.771 INFO:teuthology.orchestra.run.smithi136.stdout:finish_remove_pgs 3.cs2_head removing 3.cs2 2021-01-15T19:09:27.780 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:27.781 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:27.791 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:27.803 INFO:teuthology.orchestra.run.smithi136.stdout:Remove successful 2021-01-15T19:09:27.816 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:27.815+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:27.917 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:27.916+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:27.964 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.713070276000003, 2021-01-15T19:09:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.712879332, 2021-01-15T19:09:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.773927235000002, 2021-01-15T19:09:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.771206773999999, 2021-01-15T19:09:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.771094195000003, 2021-01-15T19:09:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.770964290999999, 2021-01-15T19:09:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.747759021999997, 2021-01-15T19:09:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.747640824000001, 2021-01-15T19:09:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.747528134, 2021-01-15T19:09:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.741926384000003, 2021-01-15T19:09:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.733457659999999, 2021-01-15T19:09:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.728939077, 2021-01-15T19:09:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.724531980000002, 2021-01-15T19:09:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.724049854, 2021-01-15T19:09:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.723921308000001, 2021-01-15T19:09:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.715472167999998, 2021-01-15T19:09:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.75679723, 2021-01-15T19:09:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.632612548000001, 2021-01-15T19:09:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.77125126, 2021-01-15T19:09:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.779610582, 2021-01-15T19:09:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:28.282 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:28.284 INFO:tasks.rados.rados.0.smithi001.stdout:399: done (8 left) 2021-01-15T19:09:28.284 INFO:tasks.rados.rados.0.smithi001.stdout:400: copy_from oid 10 from oid 44 current snap is 40 2021-01-15T19:09:28.285 INFO:tasks.rados.rados.0.smithi001.stdout:401: snap_remove snap 27 2021-01-15T19:09:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:09:28.285 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:28.282+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:28.286 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:28.287 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.cs2.6 2021-01-15T19:09:28.384 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:28.383+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:28.457 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:28.457 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:28.457 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:28.466 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:28.510 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:28.509+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:28.585 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:28.585+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:28.591 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:28.592 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:28.592 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:28.592 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:28.592 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:28.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:28.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:28.686 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:28.685+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:28.775 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:28.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:28.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.508528888000001, 2021-01-15T19:09:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:09:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:09:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:09:28.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:09:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:28.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:09:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:09:28.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:09:28.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:09:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:09:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:09:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:09:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:09:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:09:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:09:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:09:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:09:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:09:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:09:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:09:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:09:28.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:09:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:09:28.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.508117261000002, 2021-01-15T19:09:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:28.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:28.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:28.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:28.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:28.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:28.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:28.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:28.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:28.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:28.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.811 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:693 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.591531787000001, 2021-01-15T19:09:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668, 2021-01-15T19:09:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 693 2021-01-15T19:09:28.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2021-01-15T19:09:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181158+0000", 2021-01-15T19:09:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5740000000000001e-06 2021-01-15T19:09:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181165+0000", 2021-01-15T19:09:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2294e-05 2021-01-15T19:09:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181177+0000", 2021-01-15T19:09:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6979999999999998e-06 2021-01-15T19:09:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181185+0000", 2021-01-15T19:09:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.101e-05 2021-01-15T19:09:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181246+0000", 2021-01-15T19:09:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2619999999999999e-05 2021-01-15T19:09:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181258+0000", 2021-01-15T19:09:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075312950000000004 2021-01-15T19:09:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188790+0000", 2021-01-15T19:09:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1936999999999998e-05 2021-01-15T19:09:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188872+0000", 2021-01-15T19:09:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013017509999999999 2021-01-15T19:09:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.201889+0000", 2021-01-15T19:09:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668 2021-01-15T19:09:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:28.826 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.591444759000002, 2021-01-15T19:09:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.827 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.591380958999999, 2021-01-15T19:09:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:824 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.589440916000001, 2021-01-15T19:09:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001, 2021-01-15T19:09:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 824 2021-01-15T19:09:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6789000000000003e-05 2021-01-15T19:09:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183255+0000", 2021-01-15T19:09:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0429999999999996e-06 2021-01-15T19:09:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183264+0000", 2021-01-15T19:09:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7039000000000001e-05 2021-01-15T19:09:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183281+0000", 2021-01-15T19:09:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0779999999999998e-06 2021-01-15T19:09:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183288+0000", 2021-01-15T19:09:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056417399999999997 2021-01-15T19:09:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183852+0000", 2021-01-15T19:09:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086462300000000002 2021-01-15T19:09:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184717+0000", 2021-01-15T19:09:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019389661999999998 2021-01-15T19:09:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.204107+0000", 2021-01-15T19:09:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001 2021-01-15T19:09:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.588798192999999, 2021-01-15T19:09:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.588635252, 2021-01-15T19:09:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:832 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.588311139000002, 2021-01-15T19:09:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596, 2021-01-15T19:09:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 832 2021-01-15T19:09:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6316000000000003e-05 2021-01-15T19:09:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184384+0000", 2021-01-15T19:09:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:09:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184390+0000", 2021-01-15T19:09:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.133e-05 2021-01-15T19:09:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184401+0000", 2021-01-15T19:09:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7360000000000001e-06 2021-01-15T19:09:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184406+0000", 2021-01-15T19:09:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054429209999999999 2021-01-15T19:09:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189849+0000", 2021-01-15T19:09:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013384599999999999 2021-01-15T19:09:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189983+0000", 2021-01-15T19:09:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053970699999999995 2021-01-15T19:09:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.190523+0000", 2021-01-15T19:09:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099961230000000009 2021-01-15T19:09:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200519+0000", 2021-01-15T19:09:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029232770000000002 2021-01-15T19:09:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203442+0000", 2021-01-15T19:09:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8573999999999996e-05 2021-01-15T19:09:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203531+0000", 2021-01-15T19:09:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596 2021-01-15T19:09:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.427592606000001, 2021-01-15T19:09:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.427485149999999, 2021-01-15T19:09:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.294418283999999, 2021-01-15T19:09:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.398450601, 2021-01-15T19:09:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.398320471, 2021-01-15T19:09:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.398235061999999, 2021-01-15T19:09:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:28.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:28.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:28.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:28.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:28.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:28.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5670606740000004, 2021-01-15T19:09:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:09:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:29.003 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:29.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.003 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:29.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:09:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:09:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:09:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:09:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:09:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:09:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:09:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:09:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:09:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:09:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:09:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:09:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:09:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.5669720250000001, 2021-01-15T19:09:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:09:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:09:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:09:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:09:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:09:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:09:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:09:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:09:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:09:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:09:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:09:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:09:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:09:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:09:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:994 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5578979439999996, 2021-01-15T19:09:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998, 2021-01-15T19:09:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 994 2021-01-15T19:09:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3285999999999999e-05 2021-01-15T19:09:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214805+0000", 2021-01-15T19:09:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2021-01-15T19:09:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214818+0000", 2021-01-15T19:09:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5945000000000001e-05 2021-01-15T19:09:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214833+0000", 2021-01-15T19:09:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214840+0000", 2021-01-15T19:09:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9747999999999998e-05 2021-01-15T19:09:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214900+0000", 2021-01-15T19:09:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146553 2021-01-15T19:09:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215046+0000", 2021-01-15T19:09:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081407799999999998 2021-01-15T19:09:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215860+0000", 2021-01-15T19:09:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095697720000000007 2021-01-15T19:09:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225430+0000", 2021-01-15T19:09:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012440451 2021-01-15T19:09:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237871+0000", 2021-01-15T19:09:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9866999999999998e-05 2021-01-15T19:09:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237970+0000", 2021-01-15T19:09:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998 2021-01-15T19:09:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:995 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 3e] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5577781670000004, 2021-01-15T19:09:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001, 2021-01-15T19:09:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 995 2021-01-15T19:09:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8172e-05 2021-01-15T19:09:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214919+0000", 2021-01-15T19:09:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4777999999999999e-05 2021-01-15T19:09:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214944+0000", 2021-01-15T19:09:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4625e-05 2021-01-15T19:09:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214959+0000", 2021-01-15T19:09:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2830000000000001e-06 2021-01-15T19:09:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214967+0000", 2021-01-15T19:09:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002508929 2021-01-15T19:09:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217476+0000", 2021-01-15T19:09:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8955000000000006e-05 2021-01-15T19:09:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217545+0000", 2021-01-15T19:09:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066629600000000003 2021-01-15T19:09:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218211+0000", 2021-01-15T19:09:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072214000000000002 2021-01-15T19:09:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225432+0000", 2021-01-15T19:09:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012976164 2021-01-15T19:09:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238409+0000", 2021-01-15T19:09:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859000000000001e-05 2021-01-15T19:09:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238463+0000", 2021-01-15T19:09:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001 2021-01-15T19:09:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:996 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5576496820000001, 2021-01-15T19:09:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001, 2021-01-15T19:09:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 996 2021-01-15T19:09:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9251000000000001e-05 2021-01-15T19:09:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215049+0000", 2021-01-15T19:09:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:09:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215056+0000", 2021-01-15T19:09:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6064000000000001e-05 2021-01-15T19:09:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215072+0000", 2021-01-15T19:09:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:09:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215078+0000", 2021-01-15T19:09:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294249999999997 2021-01-15T19:09:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220107+0000", 2021-01-15T19:09:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4726000000000002e-05 2021-01-15T19:09:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220182+0000", 2021-01-15T19:09:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043078000000000002 2021-01-15T19:09:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220613+0000", 2021-01-15T19:09:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048209209999999997 2021-01-15T19:09:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225434+0000", 2021-01-15T19:09:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013153299 2021-01-15T19:09:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238587+0000", 2021-01-15T19:09:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5006000000000002e-05 2021-01-15T19:09:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238642+0000", 2021-01-15T19:09:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001 2021-01-15T19:09:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:29.085 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.090 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:29.091 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:29.091 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:29.092 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:29.086+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:29.092 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:29.187 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:29.186+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:29.197 INFO:tasks.rados.rados.0.smithi001.stdout:401: done (9 left) 2021-01-15T19:09:29.198 INFO:tasks.rados.rados.0.smithi001.stdout:402: rollback oid 50 current snap is 40 2021-01-15T19:09:29.198 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 50 to 33 2021-01-15T19:09:29.198 INFO:tasks.rados.rados.0.smithi001.stdout:393: expect (ObjNum 151 snap 40 seq_num 151) 2021-01-15T19:09:29.201 INFO:tasks.rados.rados.0.smithi001.stdout:391: finishing write tid 1 to smithi00140323-41 2021-01-15T19:09:29.202 INFO:tasks.rados.rados.0.smithi001.stdout:392: done (9 left) 2021-01-15T19:09:29.202 INFO:tasks.rados.rados.0.smithi001.stdout:393: done (8 left) 2021-01-15T19:09:29.202 INFO:tasks.rados.rados.0.smithi001.stdout:403: read oid 47 snap -1 2021-01-15T19:09:29.202 INFO:tasks.rados.rados.0.smithi001.stdout:403: expect deleted 2021-01-15T19:09:29.203 INFO:tasks.rados.rados.0.smithi001.stdout:404: snap_create 2021-01-15T19:09:29.203 INFO:tasks.rados.rados.0.smithi001.stdout:391: finishing write tid 2 to smithi00140323-41 2021-01-15T19:09:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:391: finishing write tid 3 to smithi00140323-41 2021-01-15T19:09:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 41 v 53 (ObjNum 154 snap 40 seq_num 154) dirty exists 2021-01-15T19:09:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:391: left oid 41 (ObjNum 154 snap 40 seq_num 154) 2021-01-15T19:09:29.204 INFO:tasks.rados.rados.0.smithi001.stdout:394: finishing write tid 1 to smithi00140323-28 2021-01-15T19:09:29.205 INFO:tasks.rados.rados.0.smithi001.stdout:394: finishing write tid 2 to smithi00140323-28 2021-01-15T19:09:29.205 INFO:tasks.rados.rados.0.smithi001.stdout:394: finishing write tid 3 to smithi00140323-28 2021-01-15T19:09:29.205 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 142 (ObjNum 155 snap 40 seq_num 155) dirty exists 2021-01-15T19:09:29.205 INFO:tasks.rados.rados.0.smithi001.stdout:394: left oid 28 (ObjNum 155 snap 40 seq_num 155) 2021-01-15T19:09:29.205 INFO:tasks.rados.rados.0.smithi001.stdout:396: finishing write tid 1 to smithi00140323-38 2021-01-15T19:09:29.206 INFO:tasks.rados.rados.0.smithi001.stdout:396: finishing write tid 2 to smithi00140323-38 2021-01-15T19:09:29.206 INFO:tasks.rados.rados.0.smithi001.stdout:396: finishing write tid 3 to smithi00140323-38 2021-01-15T19:09:29.206 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 55 (ObjNum 156 snap 40 seq_num 156) dirty exists 2021-01-15T19:09:29.207 INFO:tasks.rados.rados.0.smithi001.stdout:396: left oid 38 (ObjNum 156 snap 40 seq_num 156) 2021-01-15T19:09:29.207 INFO:tasks.rados.rados.0.smithi001.stdout:397: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:29.267 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:29.268 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:29.277 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:29.292 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:29.287+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:29.390 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:29.389+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:29.424 INFO:tasks.rados.rados.0.smithi001.stdout:400: finishing copy_from to smithi00140323-10 2021-01-15T19:09:29.424 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:29.425 INFO:tasks.rados.rados.0.smithi001.stdout:400: finishing copy_from racing read to smithi00140323-10 2021-01-15T19:09:29.425 INFO:tasks.rados.rados.0.smithi001.stdout:402: finishing rollback tid 0 to smithi00140323-50 2021-01-15T19:09:29.425 INFO:tasks.rados.rados.0.smithi001.stdout:402: finishing rollback tid 1 to smithi00140323-50 2021-01-15T19:09:29.425 INFO:tasks.rados.rados.0.smithi001.stdout:402: finishing rollback tid 2 to smithi00140323-50 2021-01-15T19:09:29.426 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 89 (ObjNum 53 snap 1 seq_num 53) dirty exists 2021-01-15T19:09:29.426 INFO:tasks.rados.rados.0.smithi001.stdout:362: finishing write tid 1 to smithi00140323-5 2021-01-15T19:09:29.426 INFO:tasks.rados.rados.0.smithi001.stdout:362: finishing write tid 2 to smithi00140323-5 2021-01-15T19:09:29.426 INFO:tasks.rados.rados.0.smithi001.stdout:362: finishing write tid 3 to smithi00140323-5 2021-01-15T19:09:29.426 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 89 (ObjNum 145 snap 36 seq_num 145) dirty exists 2021-01-15T19:09:29.427 INFO:tasks.rados.rados.0.smithi001.stdout:362: left oid 5 (ObjNum 145 snap 36 seq_num 145) 2021-01-15T19:09:29.427 INFO:tasks.rados.rados.0.smithi001.stdout:377: finishing copy_from to smithi00140323-20 2021-01-15T19:09:29.427 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 20 v 59 (ObjNum 131 snap 31 seq_num 131) dirty exists 2021-01-15T19:09:29.427 INFO:tasks.rados.rados.0.smithi001.stdout:377: finishing copy_from racing read to smithi00140323-20 2021-01-15T19:09:29.441 INFO:teuthology.orchestra.run.smithi136.stdout:get_pg_num_history pg_num_history pg_num_history(e94 pg_nums {1={11=1,58=11},2={16=8,46=18},3={21=16,52=26}} deleted_pools ) 2021-01-15T19:09:29.442 INFO:teuthology.orchestra.run.smithi136.stdout:Importing pgid 3.cs2 2021-01-15T19:09:29.442 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:30718dd9:::smithi00140323-34:36# 2021-01-15T19:09:29.446 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:30718dd9:::smithi00140323-34:head# 2021-01-15T19:09:29.446 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 36=[]:{36=[2f,2a,26,1b,19]} 2021-01-15T19:09:29.455 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:30ee460e:::smithi00140323-45:1c# 2021-01-15T19:09:29.456 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:30ee460e:::smithi00140323-45:36# 2021-01-15T19:09:29.456 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:29.456 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:29.457 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:30ee460e:::smithi00140323-45:head# 2021-01-15T19:09:29.457 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 38=[]:{1c=[1b,19],36=[2f,2a,26]} 2021-01-15T19:09:29.460 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3e4f5332:::smithi00140323-33:1b# 2021-01-15T19:09:29.461 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3e4f5332:::smithi00140323-33:28# 2021-01-15T19:09:29.464 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3e4f5332:::smithi00140323-33:2d# 2021-01-15T19:09:29.468 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3e4f5332:::smithi00140323-33:32# 2021-01-15T19:09:29.471 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:29.471 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3e4f5332:::smithi00140323-33:3a# 2021-01-15T19:09:29.474 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3e4f5332:::smithi00140323-33:head# 2021-01-15T19:09:29.476 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 3a=[]:{1b=[1b,19],28=[26],2d=[2a],32=[2f],3a=[3a,38]} 2021-01-15T19:09:29.478 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3ef45b66:::smithi00140323-11:1c# 2021-01-15T19:09:29.479 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3ef45b66:::smithi00140323-11:head# 2021-01-15T19:09:29.479 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 1c=[]:{1c=[1b,19]} 2021-01-15T19:09:29.482 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3f312ec7:::smithi00140323-7:1c# 2021-01-15T19:09:29.482 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3f312ec7:::smithi00140323-7:3a# 2021-01-15T19:09:29.484 INFO:teuthology.orchestra.run.smithi136.stdout:Write 2#3:3f312ec7:::smithi00140323-7:head# 2021-01-15T19:09:29.485 INFO:teuthology.orchestra.run.smithi136.stdout:snapset 3a=[]:{1c=[1b,19],3a=[3a,38,2f]} 2021-01-15T19:09:29.490 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:29.489+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:29.491 INFO:teuthology.orchestra.run.smithi136.stdout:write_pg epoch 93 info 3.cs2( v 91'152 (0'0,91'152] local-lis/les=86/87 n=16 ec=21/21 lis/c=86/86 les/c/f=87/87/0 sis=86) 2021-01-15T19:09:29.492 INFO:teuthology.orchestra.run.smithi136.stdout:Import successful 2021-01-15T19:09:29.592 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:29.590+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:29.604 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:29.604 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:29.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:29.693 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:29.691+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:29.781 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:29.781 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:29.791 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:29.794 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:29.793+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:29.895 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:29.894+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:29.916 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:29.916 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:29.925 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:29.997 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:29.995+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:30.010 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /var/log/ceph/exp.3.cs2.6 2021-01-15T19:09:30.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-15T19:09:30.095 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:30.095 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:30.098 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:30.097+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:30.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:30.195 INFO:tasks.rados.rados.0.smithi001.stdout:362: done (9 left) 2021-01-15T19:09:30.196 INFO:tasks.rados.rados.0.smithi001.stdout:377: done (8 left) 2021-01-15T19:09:30.196 INFO:tasks.rados.rados.0.smithi001.stdout:391: done (7 left) 2021-01-15T19:09:30.196 INFO:tasks.rados.rados.0.smithi001.stdout:394: done (6 left) 2021-01-15T19:09:30.196 INFO:tasks.rados.rados.0.smithi001.stdout:396: done (5 left) 2021-01-15T19:09:30.196 INFO:tasks.rados.rados.0.smithi001.stdout:397: done (4 left) 2021-01-15T19:09:30.197 INFO:tasks.rados.rados.0.smithi001.stdout:400: done (3 left) 2021-01-15T19:09:30.197 INFO:tasks.rados.rados.0.smithi001.stdout:402: done (2 left) 2021-01-15T19:09:30.197 INFO:tasks.rados.rados.0.smithi001.stdout:403: done (1 left) 2021-01-15T19:09:30.197 INFO:tasks.rados.rados.0.smithi001.stdout:404: done (0 left) 2021-01-15T19:09:30.197 INFO:tasks.rados.rados.0.smithi001.stdout:405: copy_from oid 43 from oid 12 current snap is 41 2021-01-15T19:09:30.198 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 35 current snap is 41 2021-01-15T19:09:30.198 INFO:tasks.rados.rados.0.smithi001.stdout:406: seq_num 157 ranges {0=237568} 2021-01-15T19:09:30.199 INFO:tasks.rados.rados.0.smithi001.stdout:406: writing smithi00140323-35 from 0 to 237568 tid 1 2021-01-15T19:09:30.199 INFO:tasks.rados.rados.0.smithi001.stdout:407: copy_from oid 16 from oid 3 current snap is 41 2021-01-15T19:09:30.200 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:30.199+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:30.200 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 22 current snap is 41 2021-01-15T19:09:30.201 INFO:tasks.rados.rados.0.smithi001.stdout:408: seq_num 158 ranges {679936=253952} 2021-01-15T19:09:30.204 INFO:tasks.rados.rados.0.smithi001.stdout:408: writing smithi00140323-22 from 679936 to 933888 tid 1 2021-01-15T19:09:30.204 INFO:tasks.rados.rados.0.smithi001.stdout:409: rollback oid 38 current snap is 41 2021-01-15T19:09:30.204 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 38 to 36 2021-01-15T19:09:30.204 INFO:tasks.rados.rados.0.smithi001.stdout:410: snap_create 2021-01-15T19:09:30.215 INFO:tasks.rados.rados.0.smithi001.stdout:408: finishing write tid 1 to smithi00140323-22 2021-01-15T19:09:30.216 INFO:tasks.rados.rados.0.smithi001.stdout:406: finishing write tid 1 to smithi00140323-35 2021-01-15T19:09:30.216 INFO:tasks.rados.rados.0.smithi001.stdout:406: finishing write tid 2 to smithi00140323-35 2021-01-15T19:09:30.217 INFO:tasks.rados.rados.0.smithi001.stdout:408: finishing write tid 2 to smithi00140323-22 2021-01-15T19:09:30.218 INFO:tasks.rados.rados.0.smithi001.stdout:406: finishing write tid 3 to smithi00140323-35 2021-01-15T19:09:30.218 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 120 (ObjNum 157 snap 41 seq_num 157) dirty exists 2021-01-15T19:09:30.218 INFO:tasks.rados.rados.0.smithi001.stdout:406: left oid 35 (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:09:30.220 INFO:tasks.rados.rados.0.smithi001.stdout:409: finishing rollback tid 0 to smithi00140323-38 2021-01-15T19:09:30.220 INFO:tasks.rados.rados.0.smithi001.stdout:409: finishing rollback tid 1 to smithi00140323-38 2021-01-15T19:09:30.221 INFO:tasks.rados.rados.0.smithi001.stdout:408: finishing write tid 3 to smithi00140323-22 2021-01-15T19:09:30.221 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 22 v 110 (ObjNum 158 snap 41 seq_num 158) dirty exists 2021-01-15T19:09:30.221 INFO:tasks.rados.rados.0.smithi001.stdout:408: left oid 22 (ObjNum 158 snap 41 seq_num 158) 2021-01-15T19:09:30.222 INFO:tasks.rados.rados.0.smithi001.stdout:409: finishing rollback tid 2 to smithi00140323-38 2021-01-15T19:09:30.222 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 58 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:30.249 INFO:tasks.rados.rados.0.smithi001.stdout:405: finishing copy_from to smithi00140323-43 2021-01-15T19:09:30.250 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 34 (ObjNum 127 snap 30 seq_num 127) dirty exists 2021-01-15T19:09:30.250 INFO:tasks.rados.rados.0.smithi001.stdout:405: finishing copy_from racing read to smithi00140323-43 2021-01-15T19:09:30.250 INFO:tasks.rados.rados.0.smithi001.stdout:407: finishing copy_from to smithi00140323-16 2021-01-15T19:09:30.251 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 38 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:30.251 INFO:tasks.rados.rados.0.smithi001.stdout:407: finishing copy_from racing read to smithi00140323-16 2021-01-15T19:09:30.262 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:30.263 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:30.273 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:30.286 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:09:30.320 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:30.318+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:30.387 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:09:30.388 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":102,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:09:30.190377+0000","last_up_change":"2021-01-15T19:09:23.192564+0000","last_in_change":"2021-01-15T19:09:17.339515+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"58","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-15T19:07:54.484638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":18,"pg_placement_num":18,"pg_placement_num_target":18,"pg_num_target":18,"pg_num_pending":18,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"46","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2021-01-15T19:08:08.242953+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":26,"pg_num_target":26,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"52","auid":0,"snap_mode":"selfmanaged","snap_seq":75,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"isaprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists","up"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists","up"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":0.198638916015625,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":95,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":34831},{"type":"v1","addr":"172.21.15.136:6810","nonce":34831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":34831},{"type":"v1","addr":"172.21.15.136:6814","nonce":34831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":34831},{"type":"v1","addr":"172.21.15.136:6822","nonce":34831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":34831},{"type":"v1","addr":"172.21.15.136:6818","nonce":34831}]},"public_addr":"172.21.15.136:6810/34831","cluster_addr":"172.21.15.136:6814/34831","heartbeat_back_addr":"172.21.15.136:6822/34831","heartbeat_front_addr":"172.21.15.136:6818/34831","state":["exists"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.060279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.065132+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.853664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.248039+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.334498+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":65536,"last_purged_snaps_scrub":"2021-01-15T19:08:49.484106+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.837613+0000","dead_epoch":95},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:08:49.696879+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"},"isaprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"2","m":"1","name":"isaprofile","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":27,"length":1},{"begin":31,"length":7},{"begin":40,"length":1},{"begin":45,"length":1},{"begin":47,"length":2},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":54,"length":1},{"begin":62,"length":1},{"begin":64,"length":1},{"begin":67,"length":1},{"begin":70,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:09:30.402 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:30.400+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:30.403 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '58', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 18, 'pg_placement_num': 18, 'pg_placement_num_target': 18, 'pg_num_target': 18, 'pg_num_pending': 18, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '46', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '52', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 75, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}] 2021-01-15T19:09:30.404 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool device_health_metrics 2021-01-15T19:09:30.415 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:30.415 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:30.426 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:30.514 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:30.513+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:30.604 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:30.604 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:30.605 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:30.602+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:30.614 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:30.705 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:30.703+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:30.785 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:30.786 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:30.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:30.843 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:30.842+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:30.907 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:30.905+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:30.931 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:30.931 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:30.941 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:31.025 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:31.024+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:31.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:31.112 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:31.113 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:31.113 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:31.111+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:31.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:31.199 INFO:tasks.rados.rados.0.smithi001.stdout:405: done (5 left) 2021-01-15T19:09:31.199 INFO:tasks.rados.rados.0.smithi001.stdout:406: done (4 left) 2021-01-15T19:09:31.199 INFO:tasks.rados.rados.0.smithi001.stdout:407: done (3 left) 2021-01-15T19:09:31.200 INFO:tasks.rados.rados.0.smithi001.stdout:408: done (2 left) 2021-01-15T19:09:31.200 INFO:tasks.rados.rados.0.smithi001.stdout:409: done (1 left) 2021-01-15T19:09:31.200 INFO:tasks.rados.rados.0.smithi001.stdout:410: done (0 left) 2021-01-15T19:09:31.200 INFO:tasks.rados.rados.0.smithi001.stdout:411: snap_remove snap 25 2021-01-15T19:09:31.202 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:09:31.209 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:31.208+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:31.245 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:31.266 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:31.286 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:31.306 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:31.310 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:31.310 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:31.310 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:31.310 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:31.311 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:31.309+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:31.320 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:31.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:31.349 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:31.370 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:31.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.372 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:31.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:31.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:31.373 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:31.374 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:31.414 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:31.413+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:31.462 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:31.463 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:31.463 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:31.463 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:31.464 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:31.472 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:31.513 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:31.511+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:31.530 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2021-01-15T19:09:31.614 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:31.613+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:31.646 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.208869847000003, 2021-01-15T19:09:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:09:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:09:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:09:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:09:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:09:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:09:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:09:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:09:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:09:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:09:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:09:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:09:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:09:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:09:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:09:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:09:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:09:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:09:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:09:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:09:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:09:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:09:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:09:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:09:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:09:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:09:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:09:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:09:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:09:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:09:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:31.686 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:31.686 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.208730547000002, 2021-01-15T19:09:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:09:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:09:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:09:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:09:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:09:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:09:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:09:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:09:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:09:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:09:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:09:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:09:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:09:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:09:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:09:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:09:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:09:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:09:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:09:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:09:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:09:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:09:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:09:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:09:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:09:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:09:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:09:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:09:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:09:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:09:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.208609441, 2021-01-15T19:09:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:09:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:09:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:09:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:09:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:09:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:09:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:09:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:09:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:09:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:09:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:09:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:09:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:09:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:09:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:09:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:09:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:09:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:09:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:09:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:09:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:09:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:09:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:09:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:09:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:09:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:09:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:09:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:09:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:09:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:09:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:09:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.438328566999999, 2021-01-15T19:09:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:09:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:09:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:09:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:09:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:09:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:09:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:09:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:09:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:09:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:09:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:09:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:09:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:09:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:09:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:09:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:09:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:09:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:09:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:09:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:09:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:09:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:09:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:09:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:09:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:09:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:09:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:09:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.429350389, 2021-01-15T19:09:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:09:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:09:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:09:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:09:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:09:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:09:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:09:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:09:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:09:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:09:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:09:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:09:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:09:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:09:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:09:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:09:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:09:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:09:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:09:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:09:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:09:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:09:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:09:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:09:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:09:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.428830689000002, 2021-01-15T19:09:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:09:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:09:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:09:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:09:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:09:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:09:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:09:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:09:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:09:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:09:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:09:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:09:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:09:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:09:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:09:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:09:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:09:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:09:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:09:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:09:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:09:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:09:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:09:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:09:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:09:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:09:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:09:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.427149735, 2021-01-15T19:09:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:09:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:09:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:09:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:09:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:09:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:09:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:09:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:09:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:09:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:09:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:09:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:09:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:09:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:09:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:09:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:09:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:09:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:09:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:09:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:09:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:09:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:09:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:09:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:09:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:09:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:09:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 47.427016780999999, 2021-01-15T19:09:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:09:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:09:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:09:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:09:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:09:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:09:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:09:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:09:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:09:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:09:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:09:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:09:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:09:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:09:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:09:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:09:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:09:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:09:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:09:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:09:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:09:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:09:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:09:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:09:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:09:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.464485510999999, 2021-01-15T19:09:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:09:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:09:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:09:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:09:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:09:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:09:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:09:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:09:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:09:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:09:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:09:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:09:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:09:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:09:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:09:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:09:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:09:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:09:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:09:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:09:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:09:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:09:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:09:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:09:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:09:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:09:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:09:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.464318917999996, 2021-01-15T19:09:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:09:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:09:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:09:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:09:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:09:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:09:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:09:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:09:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:09:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:09:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:09:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:09:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:09:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:09:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:09:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:09:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:09:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:09:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:09:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:09:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:09:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:09:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:09:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:09:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:09:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:09:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:09:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:09:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:09:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.265780123999999, 2021-01-15T19:09:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:09:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:09:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:09:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:09:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:09:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:09:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:09:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:09:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:09:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:09:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:09:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:09:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:09:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:09:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:09:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:09:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:09:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:09:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:09:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:09:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:09:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:09:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:09:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:09:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:09:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:09:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:09:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.263510079, 2021-01-15T19:09:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:09:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:09:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:09:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:09:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:09:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:09:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:09:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:09:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:09:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:09:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:09:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:09:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:09:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:09:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:09:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:09:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:09:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:09:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:09:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:09:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:09:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:09:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:09:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:09:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:09:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:09:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:09:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.258971888000001, 2021-01-15T19:09:31.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:09:31.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:09:31.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.887 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:31.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:31.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:09:31.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:09:31.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:09:31.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:09:31.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:09:31.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:09:31.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:31.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:09:31.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:09:31.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:09:31.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:09:31.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:09:31.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:09:31.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:09:31.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:09:31.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:09:31.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:09:31.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:09:31.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:09:31.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:09:31.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:09:31.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:09:31.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:09:31.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:09:31.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:09:31.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.902 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.903 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:31.903 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:31.903 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.256386016, 2021-01-15T19:09:31.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:09:31.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:09:31.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:31.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:31.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:09:31.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:09:31.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:09:31.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:09:31.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:09:31.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:09:31.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:09:31.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:09:31.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:09:31.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:09:31.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:09:31.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:09:31.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:09:31.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:09:31.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:09:31.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:09:31.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:09:31.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:09:31.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:09:31.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:09:31.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:09:31.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:09:31.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:09:31.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:09:31.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:09:31.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.920 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.921 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:31.921 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:31.921 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.256011681, 2021-01-15T19:09:31.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:09:31.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:09:31.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.923 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:31.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:31.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:09:31.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:09:31.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:09:31.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:09:31.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:09:31.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:09:31.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:09:31.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:09:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:09:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:09:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:09:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:09:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:09:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:09:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:09:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:09:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:09:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:09:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:09:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:09:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:09:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:09:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:09:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:09:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:09:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:31.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.255875186000001, 2021-01-15T19:09:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:09:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:09:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.940 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:09:31.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:09:31.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:09:31.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:09:31.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:09:31.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:09:31.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:09:31.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:09:31.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:09:31.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:09:31.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:09:31.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:31.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:09:31.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:09:31.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:09:31.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:09:31.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:31.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:09:31.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:09:31.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:09:31.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:09:31.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:09:31.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:09:31.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:09:31.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:09:31.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.955 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:889 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [copy-from ver 38] snapc 3a=[3a,38,36,2f,2a,26,25,1b,19] ondisk+write+known_if_redirected e86)", 2021-01-15T19:09:31.955 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:31.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.273587159000002, 2021-01-15T19:09:31.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999, 2021-01-15T19:09:31.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 889 2021-01-15T19:09:31.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.958 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:31.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:31.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3864e-05 2021-01-15T19:09:31.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370451+0000", 2021-01-15T19:09:31.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0156999999999999e-05 2021-01-15T19:09:31.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370461+0000", 2021-01-15T19:09:31.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2923e-05 2021-01-15T19:09:31.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370474+0000", 2021-01-15T19:09:31.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0399999999999998e-06 2021-01-15T19:09:31.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370483+0000", 2021-01-15T19:09:31.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9975999999999994e-05 2021-01-15T19:09:31.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370573+0000", 2021-01-15T19:09:31.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001672 2021-01-15T19:09:31.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370740+0000", 2021-01-15T19:09:31.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022655103999999999 2021-01-15T19:09:31.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:31.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.393395+0000", 2021-01-15T19:09:31.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087490950000000001 2021-01-15T19:09:31.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:31.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.402144+0000", 2021-01-15T19:09:31.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020087099999999999 2021-01-15T19:09:31.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:31.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404153+0000", 2021-01-15T19:09:31.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0517000000000001e-05 2021-01-15T19:09:31.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404243+0000", 2021-01-15T19:09:31.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999 2021-01-15T19:09:31.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.971 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.972 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:09:31.972 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:31.973 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.273477465999999, 2021-01-15T19:09:31.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:09:31.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:31.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:09:31.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.974 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:31.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:31.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:31.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:31.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:09:31.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:31.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:09:31.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:09:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:31.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:09:31.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:09:31.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:31.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:09:31.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:09:31.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:31.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:09:31.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:09:31.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:09:31.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:09:31.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:31.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:09:31.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:09:31.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:09:31.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:09:31.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:31.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:09:31.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:09:31.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:31.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:09:31.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:09:31.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:31.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:09:31.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:09:31.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:31.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:09:31.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:09:31.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.993 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1029 3.16s0 3:6812d466:::smithi00140323-10:head [copy-from ver 38] snapc 46=[44,41,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e100)", 2021-01-15T19:09:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:28.188310+0000", 2021-01-15T19:09:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.4556933929999998, 2021-01-15T19:09:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030765014, 2021-01-15T19:09:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1029 2021-01-15T19:09:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188310+0000", 2021-01-15T19:09:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188310+0000", 2021-01-15T19:09:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4635000000000001e-05 2021-01-15T19:09:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188365+0000", 2021-01-15T19:09:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1859999999999998e-06 2021-01-15T19:09:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188374+0000", 2021-01-15T19:09:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5869e-05 2021-01-15T19:09:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188390+0000", 2021-01-15T19:09:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3499999999999997e-06 2021-01-15T19:09:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188398+0000", 2021-01-15T19:09:32.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013117700000000001 2021-01-15T19:09:32.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.189710+0000", 2021-01-15T19:09:32.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000235577 2021-01-15T19:09:32.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.189946+0000", 2021-01-15T19:09:32.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019183228 2021-01-15T19:09:32.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.209129+0000", 2021-01-15T19:09:32.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069544350000000001 2021-01-15T19:09:32.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.216083+0000", 2021-01-15T19:09:32.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029091249999999998 2021-01-15T19:09:32.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.218993+0000", 2021-01-15T19:09:32.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2838999999999997e-05 2021-01-15T19:09:32.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219075+0000", 2021-01-15T19:09:32.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030765014 2021-01-15T19:09:32.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.014 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.014 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1030 3.16s0 3:6812d466:::smithi00140323-10:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e100)", 2021-01-15T19:09:32.015 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:28.188440+0000", 2021-01-15T19:09:32.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.45556354, 2021-01-15T19:09:32.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031043784000000001, 2021-01-15T19:09:32.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:32.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1030 2021-01-15T19:09:32.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.017 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188440+0000", 2021-01-15T19:09:32.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188440+0000", 2021-01-15T19:09:32.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7307e-05 2021-01-15T19:09:32.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188478+0000", 2021-01-15T19:09:32.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0469999999999996e-06 2021-01-15T19:09:32.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188485+0000", 2021-01-15T19:09:32.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3726e-05 2021-01-15T19:09:32.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188498+0000", 2021-01-15T19:09:32.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0440000000000001e-06 2021-01-15T19:09:32.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188504+0000", 2021-01-15T19:09:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001690932 2021-01-15T19:09:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.190195+0000", 2021-01-15T19:09:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1409999999999996e-05 2021-01-15T19:09:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.190267+0000", 2021-01-15T19:09:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025566617 2021-01-15T19:09:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.215833+0000", 2021-01-15T19:09:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0035000000000001e-05 2021-01-15T19:09:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.215923+0000", 2021-01-15T19:09:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033872780000000001 2021-01-15T19:09:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219311+0000", 2021-01-15T19:09:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7197e-05 2021-01-15T19:09:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219388+0000", 2021-01-15T19:09:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6191000000000001e-05 2021-01-15T19:09:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219484+0000", 2021-01-15T19:09:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031043784000000001 2021-01-15T19:09:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.033 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:32.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:32.036 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:32.033+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:32.052 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool rbd 2021-01-15T19:09:32.138 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:32.136+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:32.198 INFO:tasks.rados.rados.0.smithi001.stdout:411: done (0 left) 2021-01-15T19:09:32.198 INFO:tasks.rados.rados.0.smithi001.stdout:412: snap_remove snap 32 2021-01-15T19:09:32.224 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:32.226 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:32.235 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:32.236 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:32.235+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:32.338 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:32.336+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:32.366 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:32.368 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:32.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:32.460 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:32.459+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:32.540 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:32.539+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:32.550 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:32.550 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:32.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:32.551 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:32.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:32.552 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:32.552 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.29918206, 2021-01-15T19:09:32.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:32.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.553 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.553 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.553 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:32.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.554 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:32.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:32.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:32.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:32.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:32.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:32.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:32.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:32.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:32.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:32.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:32.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:32.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:32.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:32.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:32.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:32.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:32.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:32.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:32.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:32.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:32.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:32.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:32.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.570 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:32.570 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:32.570 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.298991116000003, 2021-01-15T19:09:32.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:32.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:32.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:32.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:32.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:32.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:32.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:32.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:32.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:32.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:32.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:32.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:32.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:32.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:32.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:32.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:32.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:32.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:32.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:32.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:32.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:32.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:32.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:32.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:32.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:32.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:32.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:32.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:32.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:32.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:32.591 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.592 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.592 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.593 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.593 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:32.593 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.360039018999998, 2021-01-15T19:09:32.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:32.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.594 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:32.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.595 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:32.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:32.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:32.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:32.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:32.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:32.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:32.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:32.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:32.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:32.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:32.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:32.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:32.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:32.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:32.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:32.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:32.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:32.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:32.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:32.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:32.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.610 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.611 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:32.611 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:32.611 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.357318558000003, 2021-01-15T19:09:32.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:32.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.613 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:32.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:32.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:32.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:32.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:32.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:32.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:32.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:32.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:32.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:32.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:32.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:32.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:32.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:32.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:32.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:32.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:32.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:32.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:32.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:32.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:32.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:32.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.628 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.629 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:32.629 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.357205979, 2021-01-15T19:09:32.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:32.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.630 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.631 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.631 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:32.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.631 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:32.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:32.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:32.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:32.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:32.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:32.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:32.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:32.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:32.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:32.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:32.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:32.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:32.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:32.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:32.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:32.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:32.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:32.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:32.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:32.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:32.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:32.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:32.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.645 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.646 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:32.646 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:32.646 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.357076075000002, 2021-01-15T19:09:32.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:32.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:32.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:32.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.648 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:32.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:32.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:32.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:32.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:32.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:32.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:32.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:32.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:32.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:32.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:32.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:32.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:32.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:32.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:32.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:32.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:32.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:32.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:32.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:32.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:32.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:32.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:32.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:32.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:32.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:32.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.663 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.664 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.664 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:32.664 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.333870806, 2021-01-15T19:09:32.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:32.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:32.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.667 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:32.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:32.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:32.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:32.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:32.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:32.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:32.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:32.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:32.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:32.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:32.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:32.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:32.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:32.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:32.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:32.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:32.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:32.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:32.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:32.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:32.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:32.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:32.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.680 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.681 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:32.681 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.333752607999998, 2021-01-15T19:09:32.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:32.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:32.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.683 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:32.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:32.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:32.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:32.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:32.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:32.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:32.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:32.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:32.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:32.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:32.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:32.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:32.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:32.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:32.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:32.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:32.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:32.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:32.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:32.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:32.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:32.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:32.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.697 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:32.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.333639918000003, 2021-01-15T19:09:32.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:32.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:32.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.700 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:32.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:32.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:32.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:32.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:32.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:32.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:32.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:32.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:32.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:32.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:32.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:32.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:32.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:32.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:32.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:32.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:32.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:32.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:32.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:32.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:32.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:32.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:32.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.715 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:32.715 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:32.715 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.328038167999999, 2021-01-15T19:09:32.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:32.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:32.716 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:32.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:32.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:32.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:32.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:32.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:32.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:32.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:32.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:32.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:32.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:32.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:32.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:32.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:32.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:32.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.725 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.726 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.726 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:32.727 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.319569444000003, 2021-01-15T19:09:32.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:32.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.728 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:32.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.729 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:32.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:32.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:32.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:32.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:32.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:32.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:32.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:32.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:32.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:32.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:32.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:32.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:32.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:32.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:32.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:32.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:32.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:32.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:32.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:32.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:32.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:32.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:32.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.742 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.743 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.743 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:32.744 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.315050861000003, 2021-01-15T19:09:32.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:32.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.745 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:32.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:32.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:32.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:32.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:32.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:32.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:32.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:32.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:32.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:32.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:32.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:32.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:32.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:32.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:32.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:32.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:32.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:32.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:32.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:32.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:32.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:32.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:32.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.759 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.760 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.760 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:32.760 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.310643763999998, 2021-01-15T19:09:32.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:32.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.761 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.762 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.762 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:32.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.762 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:32.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:32.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:32.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:32.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:32.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:32.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:32.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:32.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:32.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:32.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:32.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:32.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:32.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:32.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:32.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:32.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:32.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:32.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:32.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:32.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:32.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:32.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:32.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.776 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:32.777 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:32.777 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.310161637999997, 2021-01-15T19:09:32.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:32.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:32.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.779 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:32.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:32.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:32.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:32.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:32.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:32.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:32.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:32.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:32.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:32.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:32.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:32.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:32.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:32.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:32.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:32.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:32.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:32.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:32.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:32.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:32.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:32.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:32.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.793 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.794 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:32.794 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:32.794 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.310033091999998, 2021-01-15T19:09:32.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:32.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:32.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:32.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.301583952000001, 2021-01-15T19:09:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.342909014, 2021-01-15T19:09:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.218724332000001, 2021-01-15T19:09:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.357363044, 2021-01-15T19:09:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.365722366, 2021-01-15T19:09:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:32.886 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:32.888 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:32.889 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:32.886+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:32.988 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:32.986+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:33.015 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:33.015 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:33.015 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:33.015 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:33.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:33.108 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:33.106+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:33.189 INFO:tasks.rados.rados.0.smithi001.stdout:412: done (0 left) 2021-01-15T19:09:33.189 INFO:tasks.rados.rados.0.smithi001.stdout:413: copy_from oid 28 from oid 21 current snap is 42 2021-01-15T19:09:33.190 INFO:tasks.rados.rados.0.smithi001.stdout:414: read oid 35 snap -1 2021-01-15T19:09:33.190 INFO:tasks.rados.rados.0.smithi001.stdout:414: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:09:33.190 INFO:tasks.rados.rados.0.smithi001.stdout:415: snap_remove snap 13 2021-01-15T19:09:33.191 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:33.188+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:33.195 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:33.195 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:33.195 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:33.196 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:33.196 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:33.210 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:33.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:09:33.219 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2021-01-15T19:09:33.294 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:33.289+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:33.349 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:448 3.8s0 3:1009af39:::smithi00140323-25:head [copy-from ver 10] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.082778775000001, 2021-01-15T19:09:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999, 2021-01-15T19:09:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 448 2021-01-15T19:09:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264130+0000", 2021-01-15T19:09:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5605999999999998e-05 2021-01-15T19:09:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264166+0000", 2021-01-15T19:09:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264172+0000", 2021-01-15T19:09:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1133e-05 2021-01-15T19:09:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264184+0000", 2021-01-15T19:09:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0520000000000004e-06 2021-01-15T19:09:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264191+0000", 2021-01-15T19:09:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0815999999999994e-05 2021-01-15T19:09:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264261+0000", 2021-01-15T19:09:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011123700000000001 2021-01-15T19:09:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264373+0000", 2021-01-15T19:09:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012014433 2021-01-15T19:09:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.276387+0000", 2021-01-15T19:09:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061641550000000002 2021-01-15T19:09:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282551+0000", 2021-01-15T19:09:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001332724 2021-01-15T19:09:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.283884+0000", 2021-01-15T19:09:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013021699999999999 2021-01-15T19:09:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.284014+0000", 2021-01-15T19:09:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019883841999999999 2021-01-15T19:09:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.082367148000003, 2021-01-15T19:09:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:693 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.165781674000002, 2021-01-15T19:09:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668, 2021-01-15T19:09:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 693 2021-01-15T19:09:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181127+0000", 2021-01-15T19:09:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0729999999999999e-05 2021-01-15T19:09:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181158+0000", 2021-01-15T19:09:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5740000000000001e-06 2021-01-15T19:09:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181165+0000", 2021-01-15T19:09:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2294e-05 2021-01-15T19:09:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181177+0000", 2021-01-15T19:09:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6979999999999998e-06 2021-01-15T19:09:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181185+0000", 2021-01-15T19:09:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.101e-05 2021-01-15T19:09:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181246+0000", 2021-01-15T19:09:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2619999999999999e-05 2021-01-15T19:09:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181258+0000", 2021-01-15T19:09:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075312950000000004 2021-01-15T19:09:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188790+0000", 2021-01-15T19:09:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1936999999999998e-05 2021-01-15T19:09:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.188872+0000", 2021-01-15T19:09:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013017509999999999 2021-01-15T19:09:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.201889+0000", 2021-01-15T19:09:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020761668 2021-01-15T19:09:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.165694645999999, 2021-01-15T19:09:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.165630845999999, 2021-01-15T19:09:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:33.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:33.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:33.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:824 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.163690803000001, 2021-01-15T19:09:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001, 2021-01-15T19:09:33.429 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.430 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 824 2021-01-15T19:09:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183218+0000", 2021-01-15T19:09:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6789000000000003e-05 2021-01-15T19:09:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183255+0000", 2021-01-15T19:09:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0429999999999996e-06 2021-01-15T19:09:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183264+0000", 2021-01-15T19:09:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7039000000000001e-05 2021-01-15T19:09:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183281+0000", 2021-01-15T19:09:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0779999999999998e-06 2021-01-15T19:09:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183288+0000", 2021-01-15T19:09:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056417399999999997 2021-01-15T19:09:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183852+0000", 2021-01-15T19:09:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086462300000000002 2021-01-15T19:09:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184717+0000", 2021-01-15T19:09:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019389661999999998 2021-01-15T19:09:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.204107+0000", 2021-01-15T19:09:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020888408000000001 2021-01-15T19:09:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:33.441 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.163048079999999, 2021-01-15T19:09:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.442 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.443 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:33.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:33.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:33.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:33.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:33.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.162885139, 2021-01-15T19:09:33.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.465 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:832 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:33.466 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.162561025999999, 2021-01-15T19:09:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596, 2021-01-15T19:09:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.467 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 832 2021-01-15T19:09:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.468 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184348+0000", 2021-01-15T19:09:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6316000000000003e-05 2021-01-15T19:09:33.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184384+0000", 2021-01-15T19:09:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7660000000000004e-06 2021-01-15T19:09:33.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184390+0000", 2021-01-15T19:09:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.133e-05 2021-01-15T19:09:33.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184401+0000", 2021-01-15T19:09:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7360000000000001e-06 2021-01-15T19:09:33.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184406+0000", 2021-01-15T19:09:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054429209999999999 2021-01-15T19:09:33.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189849+0000", 2021-01-15T19:09:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013384599999999999 2021-01-15T19:09:33.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.189983+0000", 2021-01-15T19:09:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00053970699999999995 2021-01-15T19:09:33.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.190523+0000", 2021-01-15T19:09:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099961230000000009 2021-01-15T19:09:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.200519+0000", 2021-01-15T19:09:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029232770000000002 2021-01-15T19:09:33.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203442+0000", 2021-01-15T19:09:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8573999999999996e-05 2021-01-15T19:09:33.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.203531+0000", 2021-01-15T19:09:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019182596 2021-01-15T19:09:33.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.481 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.001842493000002, 2021-01-15T19:09:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:33.482 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.483 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.001735037, 2021-01-15T19:09:33.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.516 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:33.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.868668170999999, 2021-01-15T19:09:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.519 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:33.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:33.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:33.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:33.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:33.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:33.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:33.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:33.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:33.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:33.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:33.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:33.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.972700487999999, 2021-01-15T19:09:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:33.535 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.536 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:33.548 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.972570358, 2021-01-15T19:09:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.550 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:33.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:33.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:33.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:33.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:33.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:33.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:33.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:33.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:33.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.972484949, 2021-01-15T19:09:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:33.562 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.563 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:33.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.141310560999999, 2021-01-15T19:09:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:09:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:33.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:33.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:33.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:33.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:33.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:33.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:33.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:33.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:33.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:33.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:09:33.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:09:33.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:09:33.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:09:33.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:09:33.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:09:33.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:09:33.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:09:33.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:09:33.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:09:33.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:09:33.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:09:33.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:09:33.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.594 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.595 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:33.595 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:33.595 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.141221912000001, 2021-01-15T19:09:33.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:09:33.595 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.596 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.596 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.596 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.596 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.596 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:33.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.597 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:33.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:33.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:33.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:33.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:33.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:33.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:33.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:33.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:33.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:33.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:33.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:33.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:33.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:33.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:33.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:33.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:33.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:33.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:33.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:33.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:09:33.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:09:33.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:09:33.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:09:33.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:09:33.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:09:33.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:09:33.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:09:33.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:09:33.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:09:33.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:09:33.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:09:33.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:09:33.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:994 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:33.614 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:33.614 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1321478309999993, 2021-01-15T19:09:33.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998, 2021-01-15T19:09:33.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 994 2021-01-15T19:09:33.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:33.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:33.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3285999999999999e-05 2021-01-15T19:09:33.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214805+0000", 2021-01-15T19:09:33.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2021-01-15T19:09:33.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214818+0000", 2021-01-15T19:09:33.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5945000000000001e-05 2021-01-15T19:09:33.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214833+0000", 2021-01-15T19:09:33.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:33.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214840+0000", 2021-01-15T19:09:33.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9747999999999998e-05 2021-01-15T19:09:33.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214900+0000", 2021-01-15T19:09:33.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146553 2021-01-15T19:09:33.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215046+0000", 2021-01-15T19:09:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081407799999999998 2021-01-15T19:09:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215860+0000", 2021-01-15T19:09:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095697720000000007 2021-01-15T19:09:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225430+0000", 2021-01-15T19:09:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012440451 2021-01-15T19:09:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237871+0000", 2021-01-15T19:09:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9866999999999998e-05 2021-01-15T19:09:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237970+0000", 2021-01-15T19:09:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998 2021-01-15T19:09:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:995 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 3e] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1320280539999992, 2021-01-15T19:09:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001, 2021-01-15T19:09:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 995 2021-01-15T19:09:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8172e-05 2021-01-15T19:09:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214919+0000", 2021-01-15T19:09:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4777999999999999e-05 2021-01-15T19:09:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214944+0000", 2021-01-15T19:09:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4625e-05 2021-01-15T19:09:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214959+0000", 2021-01-15T19:09:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2830000000000001e-06 2021-01-15T19:09:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214967+0000", 2021-01-15T19:09:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002508929 2021-01-15T19:09:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217476+0000", 2021-01-15T19:09:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8955000000000006e-05 2021-01-15T19:09:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217545+0000", 2021-01-15T19:09:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066629600000000003 2021-01-15T19:09:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218211+0000", 2021-01-15T19:09:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072214000000000002 2021-01-15T19:09:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225432+0000", 2021-01-15T19:09:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012976164 2021-01-15T19:09:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238409+0000", 2021-01-15T19:09:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859000000000001e-05 2021-01-15T19:09:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238463+0000", 2021-01-15T19:09:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001 2021-01-15T19:09:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:996 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1318995689999998, 2021-01-15T19:09:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001, 2021-01-15T19:09:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 996 2021-01-15T19:09:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9251000000000001e-05 2021-01-15T19:09:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215049+0000", 2021-01-15T19:09:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:09:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215056+0000", 2021-01-15T19:09:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6064000000000001e-05 2021-01-15T19:09:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:33.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215072+0000", 2021-01-15T19:09:33.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:09:33.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215078+0000", 2021-01-15T19:09:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294249999999997 2021-01-15T19:09:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220107+0000", 2021-01-15T19:09:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4726000000000002e-05 2021-01-15T19:09:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220182+0000", 2021-01-15T19:09:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043078000000000002 2021-01-15T19:09:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220613+0000", 2021-01-15T19:09:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048209209999999997 2021-01-15T19:09:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225434+0000", 2021-01-15T19:09:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013153299 2021-01-15T19:09:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238587+0000", 2021-01-15T19:09:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5006000000000002e-05 2021-01-15T19:09:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238642+0000", 2021-01-15T19:09:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001 2021-01-15T19:09:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:33.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:33.660 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:33.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:33.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:33.663 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:33.661+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:33.751 DEBUG:teuthology.orchestra.run.smithi136:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool unique_pool_0 2021-01-15T19:09:33.763 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:33.762+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:33.796 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:33.797 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:33.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:33.810 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:33.830 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:33.850 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:33.864 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:33.862+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:33.870 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:33.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:33.912 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:33.933 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:33.933 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:33.934 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.935 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.936 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:33.936 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:33.936 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:33.937 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:33.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:33.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:33.978 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:33.977+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:34.065 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:34.063+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:34.121 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:34.122 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:34.132 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:34.167 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:34.165+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:34.195 INFO:tasks.rados.rados.0.smithi001.stdout:415: done (2 left) 2021-01-15T19:09:34.196 INFO:tasks.rados.rados.0.smithi001.stdout:416: read oid 44 snap -1 2021-01-15T19:09:34.196 INFO:tasks.rados.rados.0.smithi001.stdout:416: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:34.197 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 31 current snap is 42 2021-01-15T19:09:34.197 INFO:tasks.rados.rados.0.smithi001.stdout:417: seq_num 159 ranges {2703360=270336} 2021-01-15T19:09:34.204 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:09:34.218 INFO:tasks.rados.rados.0.smithi001.stdout:417: writing smithi00140323-31 from 2703360 to 2973696 tid 1 2021-01-15T19:09:34.218 INFO:tasks.rados.rados.0.smithi001.stdout:418: delete oid 16 current snap is 42 2021-01-15T19:09:34.219 INFO:tasks.rados.rados.0.smithi001.stdout:414: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:09:34.228 INFO:tasks.rados.rados.0.smithi001.stdout:413: finishing copy_from to smithi00140323-28 2021-01-15T19:09:34.228 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 49 (ObjNum 130 snap 31 seq_num 130) dirty exists 2021-01-15T19:09:34.228 INFO:tasks.rados.rados.0.smithi001.stdout:413: finishing copy_from racing read to smithi00140323-28 2021-01-15T19:09:34.229 INFO:tasks.rados.rados.0.smithi001.stdout:413: done (4 left) 2021-01-15T19:09:34.229 INFO:tasks.rados.rados.0.smithi001.stdout:414: done (3 left) 2021-01-15T19:09:34.229 INFO:tasks.rados.rados.0.smithi001.stdout:418: done (2 left) 2021-01-15T19:09:34.229 INFO:tasks.rados.rados.0.smithi001.stdout:419: read oid 33 snap -1 2021-01-15T19:09:34.230 INFO:tasks.rados.rados.0.smithi001.stdout:419: expect (ObjNum 138 snap 34 seq_num 138) 2021-01-15T19:09:34.230 INFO:tasks.rados.rados.0.smithi001.stdout:420: snap_create 2021-01-15T19:09:34.230 INFO:tasks.rados.rados.0.smithi001.stdout:416: expect (ObjNum 65 snap 8 seq_num 65) 2021-01-15T19:09:34.268 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:34.266+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:34.314 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:34.315 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:34.324 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:34.340 INFO:tasks.rados.rados.0.smithi001.stdout:417: finishing write tid 1 to smithi00140323-31 2021-01-15T19:09:34.341 INFO:tasks.rados.rados.0.smithi001.stdout:417: finishing write tid 2 to smithi00140323-31 2021-01-15T19:09:34.341 INFO:tasks.rados.rados.0.smithi001.stdout:417: finishing write tid 3 to smithi00140323-31 2021-01-15T19:09:34.342 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 70 (ObjNum 159 snap 42 seq_num 159) dirty exists 2021-01-15T19:09:34.342 INFO:tasks.rados.rados.0.smithi001.stdout:417: left oid 31 (ObjNum 159 snap 42 seq_num 159) 2021-01-15T19:09:34.342 INFO:tasks.rados.rados.0.smithi001.stdout:419: expect (ObjNum 138 snap 34 seq_num 138) 2021-01-15T19:09:34.380 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:34.378+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:34.471 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:34.470+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:34.509 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:34.510 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:34.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:34.571 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:34.569+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:34.651 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:34.651 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:34.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:34.705 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:34.704+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:34.713 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:34.711+0000 7fed42f4d700 -1 osd.1 pg_epoch: 106 pg[3.10s0( v 106'126 (0'0,106'126] local-lis/les=92/93 n=11 ec=52/21 lis/c=92/92 les/c/f=93/93/0 sis=92) [1,2,4]p1(0) r=0 lpr=92 crt=106'126 lcod 106'124 mlcod 106'124 active+clean+snaptrim trimq=[19~1] ps=[1b~1,2a~1,2f~1,38~1,3e~1,43~1,46~1]] removing snap head 2021-01-15T19:09:34.773 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:34.772+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:34.798 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:34.798 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:34.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:34.848 INFO:teuthology.orchestra.run.smithi136.stderr:Nothing to do for non-filestore backend 2021-01-15T19:09:34.874 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:34.873+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:34.975 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:34.974+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:34.978 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:34.979 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:34.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:35.076 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:35.074+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:35.120 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:35.120 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:35.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:35.214 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:35.213+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:35.216 INFO:tasks.rados.rados.0.smithi001.stdout:416: done (3 left) 2021-01-15T19:09:35.217 INFO:tasks.rados.rados.0.smithi001.stdout:417: done (2 left) 2021-01-15T19:09:35.217 INFO:tasks.rados.rados.0.smithi001.stdout:419: done (1 left) 2021-01-15T19:09:35.217 INFO:tasks.rados.rados.0.smithi001.stdout:420: done (0 left) 2021-01-15T19:09:35.217 INFO:tasks.rados.rados.0.smithi001.stdout:421: snap_remove snap 41 2021-01-15T19:09:35.278 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:35.276+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:35.304 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:35.304 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:35.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:35.411 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:35.409+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:35.480 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:35.478+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:35.485 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:35.485 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:35.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:35.581 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:35.580+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:35.627 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:35.628 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:35.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:35.682 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:35.681+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:35.769 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:35.770 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:35.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:35.820 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:35.819+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:35.884 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:35.883+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:35.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:35.951 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:35.951 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:35.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:36.004 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:36.003+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:36.087 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:36.086+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:36.088 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:36.109 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:36.130 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:36.142 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:36.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:36.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.704453133000001, 2021-01-15T19:09:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:09:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:09:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.145 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:09:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:09:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:09:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:09:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:09:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:09:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:09:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:09:36.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:09:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:09:36.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:09:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:09:36.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:09:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:09:36.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:09:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:09:36.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:09:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:09:36.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:09:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:09:36.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:09:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:09:36.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:09:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:09:36.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:09:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:09:36.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:09:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:09:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:09:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:36.164 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.704313833, 2021-01-15T19:09:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:09:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:09:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.166 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:09:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:09:36.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:09:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:36.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:09:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:09:36.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:09:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:09:36.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:09:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:09:36.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:09:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:09:36.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:09:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:09:36.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:09:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:09:36.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:09:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:09:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:09:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:09:36.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:09:36.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:09:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:09:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:09:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:09:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:09:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:09:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:09:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:09:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:09:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.185 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.704192726999999, 2021-01-15T19:09:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:09:36.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:09:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:09:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:09:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:09:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:09:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:09:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:09:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:09:36.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:09:36.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:09:36.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:09:36.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:09:36.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:09:36.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:09:36.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:09:36.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:09:36.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:09:36.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:09:36.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:09:36.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:09:36.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:09:36.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:09:36.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:09:36.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:09:36.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:09:36.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:09:36.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:09:36.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:09:36.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:09:36.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:09:36.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:09:36.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.206 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.933911852999998, 2021-01-15T19:09:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:09:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:09:36.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.209 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:36.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:09:36.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:09:36.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:09:36.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:09:36.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:09:36.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:09:36.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:09:36.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:09:36.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:09:36.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:09:36.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:09:36.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:09:36.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:09:36.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:09:36.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:09:36.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:09:36.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:09:36.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:09:36.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:09:36.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:09:36.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:09:36.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:09:36.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:09:36.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:09:36.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:09:36.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:09:36.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.224 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.225 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:36.225 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:36.225 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.924933674999998, 2021-01-15T19:09:36.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:09:36.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:09:36.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.227 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:36.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:09:36.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:09:36.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:09:36.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:09:36.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:09:36.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:09:36.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:09:36.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:36.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:09:36.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:09:36.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:36.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:09:36.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:09:36.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:09:36.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:09:36.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:09:36.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:09:36.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:09:36.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:09:36.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:09:36.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:09:36.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:09:36.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:09:36.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:09:36.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:09:36.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:09:36.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:36.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:36.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.924413975, 2021-01-15T19:09:36.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:09:36.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:09:36.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.245 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:09:36.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:09:36.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:09:36.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:09:36.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:09:36.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:09:36.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:09:36.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:09:36.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:09:36.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:09:36.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:09:36.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:09:36.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:09:36.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:09:36.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:09:36.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:09:36.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:09:36.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:09:36.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:09:36.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:09:36.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:09:36.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:09:36.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:09:36.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:09:36.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:09:36.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:09:36.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.261 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.261 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.261 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:36.262 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:36.262 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.922733020999999, 2021-01-15T19:09:36.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:09:36.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.263 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:09:36.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.264 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:36.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:09:36.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:09:36.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:09:36.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:09:36.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:09:36.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:09:36.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:09:36.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:09:36.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:09:36.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:09:36.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:36.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:09:36.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:09:36.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:09:36.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:09:36.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:09:36.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:09:36.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:09:36.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:09:36.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:09:36.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:09:36.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:09:36.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:09:36.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:09:36.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:09:36.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:09:36.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.279 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.280 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:36.280 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:36.280 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.922600066999998, 2021-01-15T19:09:36.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:09:36.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:36.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.282 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:09:36.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.282 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:36.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:09:36.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:09:36.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:09:36.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:09:36.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:09:36.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:09:36.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:09:36.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:09:36.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:09:36.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:09:36.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:09:36.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:09:36.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:36.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:09:36.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:09:36.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:09:36.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:09:36.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:36.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:09:36.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:09:36.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:09:36.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:09:36.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:09:36.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:09:36.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:09:36.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:09:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.301 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.960068796999998, 2021-01-15T19:09:36.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:09:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:09:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:09:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:09:36.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:09:36.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:09:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:09:36.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:09:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:09:36.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:09:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:09:36.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:09:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:09:36.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:09:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:36.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:09:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:09:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:09:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:09:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:09:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:09:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:09:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:09:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:09:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:09:36.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:09:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:09:36.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:09:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:09:36.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.320 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.959902204000002, 2021-01-15T19:09:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:09:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:09:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:09:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:09:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:09:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:09:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:09:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:09:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:09:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:09:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:09:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:09:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:09:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:09:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:36.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:09:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:09:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:09:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:09:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:36.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:09:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:09:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:09:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:09:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:36.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:09:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:09:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:09:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:09:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:09:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:09:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:09:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:09:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:36.339 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.761363410000001, 2021-01-15T19:09:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:09:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.340 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:09:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:09:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:09:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:09:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:09:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:09:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:09:36.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:09:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:09:36.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:09:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:09:36.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:09:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:09:36.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:09:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:09:36.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:09:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:09:36.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:09:36.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:09:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:09:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:09:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:09:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:09:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:09:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:09:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:09:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:09:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.759093364999998, 2021-01-15T19:09:36.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:09:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.358 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:09:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:09:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:09:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:09:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:09:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:09:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:09:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:09:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:09:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:09:36.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:09:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:09:36.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:09:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:09:36.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:09:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:09:36.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:09:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:09:36.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:09:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:09:36.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:09:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:09:36.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:09:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:09:36.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:09:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:09:36.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:09:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.754555174, 2021-01-15T19:09:36.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:09:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.376 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:09:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:09:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:09:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:09:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:09:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:09:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:09:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:09:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:09:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:09:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:09:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:09:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:09:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:09:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:09:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:09:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:09:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:09:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:09:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:09:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:09:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:09:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:09:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:09:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:09:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:09:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:09:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.391 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:36.392 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.751969301999999, 2021-01-15T19:09:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:09:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.393 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:09:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.394 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:09:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:09:36.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:09:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:09:36.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:09:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:09:36.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:09:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:09:36.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:09:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:09:36.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:09:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:09:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:09:36.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:09:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:09:36.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:09:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:09:36.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:09:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:09:36.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:09:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:09:36.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:09:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:09:36.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:09:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:09:36.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:09:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.751594966999999, 2021-01-15T19:09:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:09:36.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:09:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:09:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:09:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:09:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:09:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:09:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:09:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:09:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:09:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:09:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:09:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:09:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:09:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:09:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:09:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:09:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:09:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:09:36.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:09:36.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:09:36.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:09:36.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:09:36.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:09:36.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:09:36.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:09:36.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:09:36.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:09:36.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.426 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:36.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:36.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.751458471999999, 2021-01-15T19:09:36.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:09:36.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:36.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:09:36.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.429 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:36.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:09:36.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:09:36.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:09:36.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:09:36.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:09:36.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:09:36.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:09:36.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:09:36.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:09:36.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:09:36.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:09:36.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:09:36.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:36.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:09:36.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:09:36.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:09:36.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:09:36.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:36.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:09:36.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:09:36.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:09:36.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:09:36.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:09:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:09:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:09:36.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:09:36.442 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.442 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.443 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:889 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [copy-from ver 38] snapc 3a=[3a,38,36,2f,2a,26,25,1b,19] ondisk+write+known_if_redirected e86)", 2021-01-15T19:09:36.443 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:36.444 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.769170445, 2021-01-15T19:09:36.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999, 2021-01-15T19:09:36.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.444 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 889 2021-01-15T19:09:36.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.445 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:36.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370417+0000", 2021-01-15T19:09:36.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3864e-05 2021-01-15T19:09:36.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370451+0000", 2021-01-15T19:09:36.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0156999999999999e-05 2021-01-15T19:09:36.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370461+0000", 2021-01-15T19:09:36.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2923e-05 2021-01-15T19:09:36.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370474+0000", 2021-01-15T19:09:36.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0399999999999998e-06 2021-01-15T19:09:36.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370483+0000", 2021-01-15T19:09:36.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9975999999999994e-05 2021-01-15T19:09:36.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370573+0000", 2021-01-15T19:09:36.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0001672 2021-01-15T19:09:36.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370740+0000", 2021-01-15T19:09:36.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022655103999999999 2021-01-15T19:09:36.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.393395+0000", 2021-01-15T19:09:36.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087490950000000001 2021-01-15T19:09:36.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.402144+0000", 2021-01-15T19:09:36.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020087099999999999 2021-01-15T19:09:36.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404153+0000", 2021-01-15T19:09:36.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0517000000000001e-05 2021-01-15T19:09:36.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404243+0000", 2021-01-15T19:09:36.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.033826585999999999 2021-01-15T19:09:36.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.459 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:09:36.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:36.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 21.769060752000001, 2021-01-15T19:09:36.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:09:36.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:36.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:09:36.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.461 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:36.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:09:36.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:09:36.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:09:36.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:09:36.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:09:36.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:09:36.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:09:36.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:09:36.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:09:36.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:09:36.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:09:36.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:09:36.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:09:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:09:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:09:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:09:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:36.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:09:36.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:09:36.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:09:36.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:09:36.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:09:36.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:09:36.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:09:36.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:09:36.474 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.474 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.475 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1029 3.16s0 3:6812d466:::smithi00140323-10:head [copy-from ver 38] snapc 46=[44,41,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e100)", 2021-01-15T19:09:36.475 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:28.188310+0000", 2021-01-15T19:09:36.476 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.9512766790000002, 2021-01-15T19:09:36.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030765014, 2021-01-15T19:09:36.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:36.476 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.477 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1029 2021-01-15T19:09:36.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.477 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188310+0000", 2021-01-15T19:09:36.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188310+0000", 2021-01-15T19:09:36.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4635000000000001e-05 2021-01-15T19:09:36.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188365+0000", 2021-01-15T19:09:36.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1859999999999998e-06 2021-01-15T19:09:36.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188374+0000", 2021-01-15T19:09:36.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5869e-05 2021-01-15T19:09:36.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188390+0000", 2021-01-15T19:09:36.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3499999999999997e-06 2021-01-15T19:09:36.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188398+0000", 2021-01-15T19:09:36.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013117700000000001 2021-01-15T19:09:36.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.189710+0000", 2021-01-15T19:09:36.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000235577 2021-01-15T19:09:36.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.189946+0000", 2021-01-15T19:09:36.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019183228 2021-01-15T19:09:36.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:36.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.209129+0000", 2021-01-15T19:09:36.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069544350000000001 2021-01-15T19:09:36.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:36.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.216083+0000", 2021-01-15T19:09:36.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0029091249999999998 2021-01-15T19:09:36.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:36.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.218993+0000", 2021-01-15T19:09:36.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2838999999999997e-05 2021-01-15T19:09:36.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219075+0000", 2021-01-15T19:09:36.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030765014 2021-01-15T19:09:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.491 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1030 3.16s0 3:6812d466:::smithi00140323-10:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e100)", 2021-01-15T19:09:36.491 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:28.188440+0000", 2021-01-15T19:09:36.491 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 7.9511468259999996, 2021-01-15T19:09:36.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031043784000000001, 2021-01-15T19:09:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:36.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1030 2021-01-15T19:09:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188440+0000", 2021-01-15T19:09:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:36.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188440+0000", 2021-01-15T19:09:36.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7307e-05 2021-01-15T19:09:36.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:36.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188478+0000", 2021-01-15T19:09:36.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0469999999999996e-06 2021-01-15T19:09:36.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:36.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188485+0000", 2021-01-15T19:09:36.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3726e-05 2021-01-15T19:09:36.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:36.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188498+0000", 2021-01-15T19:09:36.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0440000000000001e-06 2021-01-15T19:09:36.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:36.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.188504+0000", 2021-01-15T19:09:36.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001690932 2021-01-15T19:09:36.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.190195+0000", 2021-01-15T19:09:36.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1409999999999996e-05 2021-01-15T19:09:36.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:36.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.190267+0000", 2021-01-15T19:09:36.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025566617 2021-01-15T19:09:36.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.215833+0000", 2021-01-15T19:09:36.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0035000000000001e-05 2021-01-15T19:09:36.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:36.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.215923+0000", 2021-01-15T19:09:36.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033872780000000001 2021-01-15T19:09:36.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:36.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219311+0000", 2021-01-15T19:09:36.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7197e-05 2021-01-15T19:09:36.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:36.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219388+0000", 2021-01-15T19:09:36.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6191000000000001e-05 2021-01-15T19:09:36.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:36.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:36.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:36.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:28.219484+0000", 2021-01-15T19:09:36.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.031043784000000001 2021-01-15T19:09:36.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.506 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:36.507 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:36.507 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:36.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:09:36.511 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:36.511 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:36.511 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:36.511 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:36.512 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:36.512 INFO:tasks.rados.rados.0.smithi001.stdout:421: done (0 left) 2021-01-15T19:09:36.513 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 44 current snap is 43 2021-01-15T19:09:36.513 INFO:tasks.rados.rados.0.smithi001.stdout:422: seq_num 161 ranges {2703360=303104} 2021-01-15T19:09:36.513 INFO:tasks.rados.rados.0.smithi001.stdout:422: writing smithi00140323-44 from 2703360 to 3006464 tid 1 2021-01-15T19:09:36.513 INFO:tasks.rados.rados.0.smithi001.stdout:423: delete oid 3 current snap is 43 2021-01-15T19:09:36.514 INFO:tasks.rados.rados.0.smithi001.stdout:422: finishing write tid 1 to smithi00140323-44 2021-01-15T19:09:36.514 INFO:tasks.rados.rados.0.smithi001.stdout:422: finishing write tid 2 to smithi00140323-44 2021-01-15T19:09:36.514 INFO:tasks.rados.rados.0.smithi001.stdout:422: finishing write tid 3 to smithi00140323-44 2021-01-15T19:09:36.514 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 44 v 43 (ObjNum 161 snap 43 seq_num 161) dirty exists 2021-01-15T19:09:36.514 INFO:tasks.rados.rados.0.smithi001.stdout:422: left oid 44 (ObjNum 161 snap 43 seq_num 161) 2021-01-15T19:09:36.515 INFO:tasks.rados.rados.0.smithi001.stdout:422: done (1 left) 2021-01-15T19:09:36.515 INFO:tasks.rados.rados.0.smithi001.stdout:423: done (0 left) 2021-01-15T19:09:36.515 INFO:tasks.rados.rados.0.smithi001.stdout:424: copy_from oid 4 from oid 11 current snap is 43 2021-01-15T19:09:36.515 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 5 current snap is 43 2021-01-15T19:09:36.515 INFO:tasks.rados.rados.0.smithi001.stdout:425: seq_num 163 ranges {352256=335872} 2021-01-15T19:09:36.516 INFO:tasks.rados.rados.0.smithi001.stdout:425: writing smithi00140323-5 from 352256 to 688128 tid 1 2021-01-15T19:09:36.516 INFO:tasks.rados.rados.0.smithi001.stdout:426: setattr oid 42 current snap is 43 2021-01-15T19:09:36.516 INFO:tasks.rados.rados.0.smithi001.stdout:427: setattr oid 8 current snap is 43 2021-01-15T19:09:36.516 INFO:tasks.rados.rados.0.smithi001.stdout:428: copy_from oid 49 from oid 29 current snap is 43 2021-01-15T19:09:36.516 INFO:tasks.rados.rados.0.smithi001.stdout:429: snap_remove snap 33 2021-01-15T19:09:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.518 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.518 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:36.518 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:36.518 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:36.519 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:36.519 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:36.519 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:36.506+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:36.611 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:36.610+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:36.686 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:36.686 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:36.686 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:36.687 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:36.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:36.714 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:36.711+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:36.814 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:36.812+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:36.868 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:36.868 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:36.869 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:36.869 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:36.878 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:36.923 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:36.922+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:37.011 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.760231556000001, 2021-01-15T19:09:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.760040611999997, 2021-01-15T19:09:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.821088515, 2021-01-15T19:09:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.818368053999997, 2021-01-15T19:09:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.818255475000001, 2021-01-15T19:09:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.818125571000003, 2021-01-15T19:09:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.794920302000001, 2021-01-15T19:09:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.794802103999999, 2021-01-15T19:09:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.794689413999997, 2021-01-15T19:09:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.789087664, 2021-01-15T19:09:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.780618939999997, 2021-01-15T19:09:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.776100356999997, 2021-01-15T19:09:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.771693259999999, 2021-01-15T19:09:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.771211133999998, 2021-01-15T19:09:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.771082587999999, 2021-01-15T19:09:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 52.762633448000003, 2021-01-15T19:09:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.803958510000001, 2021-01-15T19:09:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.679773828000002, 2021-01-15T19:09:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 32.818412539999997, 2021-01-15T19:09:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.826771862000001, 2021-01-15T19:09:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:37.337 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:37.339 INFO:tasks.rados.rados.0.smithi001.stdout:429: done (5 left) 2021-01-15T19:09:37.340 INFO:tasks.rados.rados.0.smithi001.stdout:430: rmattr oid 22 current snap is 43 2021-01-15T19:09:37.340 INFO:tasks.rados.rados.0.smithi001.stdout:430: done (5 left) 2021-01-15T19:09:37.340 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 43 2021-01-15T19:09:37.340 INFO:tasks.rados.rados.0.smithi001.stdout:431: seq_num 164 ranges {0=352256} 2021-01-15T19:09:37.341 INFO:tasks.rados.rados.0.smithi001.stdout:431: writing smithi00140323-39 from 0 to 352256 tid 1 2021-01-15T19:09:37.341 INFO:tasks.rados.rados.0.smithi001.stdout:432: rollback oid 24 current snap is 43 2021-01-15T19:09:37.341 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 24 to 36 2021-01-15T19:09:37.341 INFO:tasks.rados.rados.0.smithi001.stdout:433: rollback oid 47 current snap is 43 2021-01-15T19:09:37.341 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 47 to 42 2021-01-15T19:09:37.342 INFO:tasks.rados.rados.0.smithi001.stdout:434: snap_create 2021-01-15T19:09:37.342 INFO:tasks.rados.rados.0.smithi001.stdout:425: finishing write tid 1 to smithi00140323-5 2021-01-15T19:09:37.342 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 52 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:09:37.342 INFO:tasks.rados.rados.0.smithi001.stdout:425: finishing write tid 2 to smithi00140323-5 2021-01-15T19:09:37.343 INFO:tasks.rados.rados.0.smithi001.stdout:425: finishing write tid 3 to smithi00140323-5 2021-01-15T19:09:37.343 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 112 (ObjNum 163 snap 43 seq_num 163) dirty exists 2021-01-15T19:09:37.343 INFO:tasks.rados.rados.0.smithi001.stdout:425: left oid 5 (ObjNum 163 snap 43 seq_num 163) 2021-01-15T19:09:37.343 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 111 (ObjNum 131 snap 31 seq_num 131) dirty exists 2021-01-15T19:09:37.343 INFO:tasks.rados.rados.0.smithi001.stdout:424: finishing copy_from to smithi00140323-4 2021-01-15T19:09:37.344 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 67 (ObjNum 102 snap 16 seq_num 102) dirty exists 2021-01-15T19:09:37.344 INFO:tasks.rados.rados.0.smithi001.stdout:424: finishing copy_from racing read to smithi00140323-4 2021-01-15T19:09:37.344 INFO:tasks.rados.rados.0.smithi001.stdout:428: finishing copy_from to smithi00140323-49 2021-01-15T19:09:37.344 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 72 (ObjNum 122 snap 27 seq_num 122) dirty exists 2021-01-15T19:09:37.344 INFO:tasks.rados.rados.0.smithi001.stdout:428: finishing copy_from racing read to smithi00140323-49 2021-01-15T19:09:37.345 INFO:tasks.rados.rados.0.smithi001.stdout:431: finishing write tid 1 to smithi00140323-39 2021-01-15T19:09:37.345 INFO:tasks.rados.rados.0.smithi001.stdout:431: finishing write tid 2 to smithi00140323-39 2021-01-15T19:09:37.345 INFO:tasks.rados.rados.0.smithi001.stdout:431: finishing write tid 3 to smithi00140323-39 2021-01-15T19:09:37.345 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 50 (ObjNum 164 snap 43 seq_num 164) dirty exists 2021-01-15T19:09:37.345 INFO:tasks.rados.rados.0.smithi001.stdout:431: left oid 39 (ObjNum 164 snap 43 seq_num 164) 2021-01-15T19:09:37.346 INFO:tasks.rados.rados.0.smithi001.stdout:433: finishing rollback tid 1 to smithi00140323-47 2021-01-15T19:09:37.346 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 99 (ObjNum 178878560 snap 22026 seq_num 178954800) dirty dne 2021-01-15T19:09:37.346 INFO:tasks.rados.rados.0.smithi001.stdout:432: finishing rollback tid 0 to smithi00140323-24 2021-01-15T19:09:37.346 INFO:tasks.rados.rados.0.smithi001.stdout:432: finishing rollback tid 1 to smithi00140323-24 2021-01-15T19:09:37.346 INFO:tasks.rados.rados.0.smithi001.stdout:432: finishing rollback tid 2 to smithi00140323-24 2021-01-15T19:09:37.347 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 24 v 62 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:09:37.347 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:37.337+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:37.348 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:37.439 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:37.438+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:37.483 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:37.483 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:37.484 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:37.484 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:37.494 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:37.540 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:37.539+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:37.619 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:37.619 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:37.620 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:37.620 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:37.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:37.641 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:37.640+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:37.743 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:37.740+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:37.759 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.491745068999997, 2021-01-15T19:09:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.575072566999999, 2021-01-15T19:09:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.575008767, 2021-01-15T19:09:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.572426001, 2021-01-15T19:09:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.572263060000001, 2021-01-15T19:09:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.411220413999999, 2021-01-15T19:09:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.411112958, 2021-01-15T19:09:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.278046092, 2021-01-15T19:09:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.382078408999998, 2021-01-15T19:09:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.381948279, 2021-01-15T19:09:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.381862869999999, 2021-01-15T19:09:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.550688482, 2021-01-15T19:09:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:09:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:09:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:09:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:09:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:09:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:09:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:09:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:09:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:09:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:09:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:09:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:09:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:09:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:09:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.550599833, 2021-01-15T19:09:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:09:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:09:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:09:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:09:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:09:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:09:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:09:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:09:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:09:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:09:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:09:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:09:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:09:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:09:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:994 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.541525752, 2021-01-15T19:09:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998, 2021-01-15T19:09:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 994 2021-01-15T19:09:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3285999999999999e-05 2021-01-15T19:09:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214805+0000", 2021-01-15T19:09:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2021-01-15T19:09:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214818+0000", 2021-01-15T19:09:38.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5945000000000001e-05 2021-01-15T19:09:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214833+0000", 2021-01-15T19:09:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214840+0000", 2021-01-15T19:09:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9747999999999998e-05 2021-01-15T19:09:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214900+0000", 2021-01-15T19:09:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146553 2021-01-15T19:09:38.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215046+0000", 2021-01-15T19:09:38.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081407799999999998 2021-01-15T19:09:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215860+0000", 2021-01-15T19:09:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095697720000000007 2021-01-15T19:09:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225430+0000", 2021-01-15T19:09:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012440451 2021-01-15T19:09:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237871+0000", 2021-01-15T19:09:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9866999999999998e-05 2021-01-15T19:09:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237970+0000", 2021-01-15T19:09:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998 2021-01-15T19:09:38.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:995 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 3e] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.541405975, 2021-01-15T19:09:38.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001, 2021-01-15T19:09:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 995 2021-01-15T19:09:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.044 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:38.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8172e-05 2021-01-15T19:09:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214919+0000", 2021-01-15T19:09:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4777999999999999e-05 2021-01-15T19:09:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214944+0000", 2021-01-15T19:09:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4625e-05 2021-01-15T19:09:38.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214959+0000", 2021-01-15T19:09:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2830000000000001e-06 2021-01-15T19:09:38.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214967+0000", 2021-01-15T19:09:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002508929 2021-01-15T19:09:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217476+0000", 2021-01-15T19:09:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8955000000000006e-05 2021-01-15T19:09:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217545+0000", 2021-01-15T19:09:38.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066629600000000003 2021-01-15T19:09:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218211+0000", 2021-01-15T19:09:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072214000000000002 2021-01-15T19:09:38.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225432+0000", 2021-01-15T19:09:38.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012976164 2021-01-15T19:09:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238409+0000", 2021-01-15T19:09:38.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859000000000001e-05 2021-01-15T19:09:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238463+0000", 2021-01-15T19:09:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001 2021-01-15T19:09:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.061 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:996 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:38.063 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.541277490000001, 2021-01-15T19:09:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001, 2021-01-15T19:09:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.064 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 996 2021-01-15T19:09:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:38.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9251000000000001e-05 2021-01-15T19:09:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215049+0000", 2021-01-15T19:09:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:09:38.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215056+0000", 2021-01-15T19:09:38.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6064000000000001e-05 2021-01-15T19:09:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215072+0000", 2021-01-15T19:09:38.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:09:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215078+0000", 2021-01-15T19:09:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294249999999997 2021-01-15T19:09:38.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220107+0000", 2021-01-15T19:09:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4726000000000002e-05 2021-01-15T19:09:38.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220182+0000", 2021-01-15T19:09:38.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043078000000000002 2021-01-15T19:09:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220613+0000", 2021-01-15T19:09:38.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048209209999999997 2021-01-15T19:09:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225434+0000", 2021-01-15T19:09:38.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013153299 2021-01-15T19:09:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238587+0000", 2021-01-15T19:09:38.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5006000000000002e-05 2021-01-15T19:09:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238642+0000", 2021-01-15T19:09:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001 2021-01-15T19:09:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:09:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5670917749999997, 2021-01-15T19:09:38.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:09:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:09:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:09:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:09:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:09:38.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:09:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:09:38.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:09:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:09:38.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:09:38.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:09:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:09:38.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:09:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:09:38.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:09:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:09:38.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:09:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:09:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:09:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:09:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:09:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:09:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:09:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:09:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:09:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:09:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:09:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:09:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5669597, 2021-01-15T19:09:38.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:09:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:09:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:09:38.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:09:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:38.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:09:38.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:09:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:09:38.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:09:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:09:38.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:09:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:09:38.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:09:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:09:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:09:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:09:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:09:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:38.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:09:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:09:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:09:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:09:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:09:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:09:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:09:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:09:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:09:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:09:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:09:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:09:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1084 3.6s0 3:660240bf:::smithi00140323-49:head [copy-from ver 72] snapc 50=[50,4b,44,41,3a,26] ondisk+write+known_if_redirected e108)", 2021-01-15T19:09:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:36.255451+0000", 2021-01-15T19:09:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5008360759999999, 2021-01-15T19:09:38.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021142780999999999, 2021-01-15T19:09:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1084 2021-01-15T19:09:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.121 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255451+0000", 2021-01-15T19:09:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255451+0000", 2021-01-15T19:09:38.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9991000000000001e-05 2021-01-15T19:09:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255481+0000", 2021-01-15T19:09:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8889999999999999e-06 2021-01-15T19:09:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255487+0000", 2021-01-15T19:09:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0183999999999999e-05 2021-01-15T19:09:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255497+0000", 2021-01-15T19:09:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.541e-06 2021-01-15T19:09:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255503+0000", 2021-01-15T19:09:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5642000000000006e-05 2021-01-15T19:09:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255568+0000", 2021-01-15T19:09:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132068 2021-01-15T19:09:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255700+0000", 2021-01-15T19:09:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014169932 2021-01-15T19:09:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.269870+0000", 2021-01-15T19:09:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061704660000000003 2021-01-15T19:09:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.276041+0000", 2021-01-15T19:09:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010785000000000001 2021-01-15T19:09:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.276149+0000", 2021-01-15T19:09:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044521799999999999 2021-01-15T19:09:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.276594+0000", 2021-01-15T19:09:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021142780999999999 2021-01-15T19:09:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.134 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1085 3.6s0 3:660240bf:::smithi00140323-49:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e108)", 2021-01-15T19:09:38.135 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:36.255555+0000", 2021-01-15T19:09:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.500731882, 2021-01-15T19:09:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021321848000000001, 2021-01-15T19:09:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.136 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1085 2021-01-15T19:09:38.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255555+0000", 2021-01-15T19:09:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255555+0000", 2021-01-15T19:09:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3292e-05 2021-01-15T19:09:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255578+0000", 2021-01-15T19:09:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2429999999999999e-06 2021-01-15T19:09:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255583+0000", 2021-01-15T19:09:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1839999999999999e-06 2021-01-15T19:09:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255591+0000", 2021-01-15T19:09:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9790000000000004e-06 2021-01-15T19:09:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255595+0000", 2021-01-15T19:09:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026501900000000002 2021-01-15T19:09:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255860+0000", 2021-01-15T19:09:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7567e-05 2021-01-15T19:09:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.255907+0000", 2021-01-15T19:09:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019832552999999999 2021-01-15T19:09:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.275740+0000", 2021-01-15T19:09:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7683000000000006e-05 2021-01-15T19:09:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.275838+0000", 2021-01-15T19:09:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000796878 2021-01-15T19:09:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.276635+0000", 2021-01-15T19:09:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114943 2021-01-15T19:09:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.276749+0000", 2021-01-15T19:09:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012750700000000001 2021-01-15T19:09:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:36.276877+0000", 2021-01-15T19:09:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021321848000000001 2021-01-15T19:09:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.151 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.152 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:38.154 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:38.152+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:38.154 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:38.190 INFO:tasks.rados.rados.0.smithi001.stdout:424: done (8 left) 2021-01-15T19:09:38.191 INFO:tasks.rados.rados.0.smithi001.stdout:425: done (7 left) 2021-01-15T19:09:38.191 INFO:tasks.rados.rados.0.smithi001.stdout:426: done (6 left) 2021-01-15T19:09:38.191 INFO:tasks.rados.rados.0.smithi001.stdout:427: done (5 left) 2021-01-15T19:09:38.192 INFO:tasks.rados.rados.0.smithi001.stdout:428: done (4 left) 2021-01-15T19:09:38.192 INFO:tasks.rados.rados.0.smithi001.stdout:431: done (3 left) 2021-01-15T19:09:38.192 INFO:tasks.rados.rados.0.smithi001.stdout:432: done (2 left) 2021-01-15T19:09:38.192 INFO:tasks.rados.rados.0.smithi001.stdout:433: done (1 left) 2021-01-15T19:09:38.193 INFO:tasks.rados.rados.0.smithi001.stdout:434: done (0 left) 2021-01-15T19:09:38.193 INFO:tasks.rados.rados.0.smithi001.stdout:435: setattr oid 7 current snap is 44 2021-01-15T19:09:38.194 INFO:tasks.rados.rados.0.smithi001.stdout:436: copy_from oid 43 from oid 12 current snap is 44 2021-01-15T19:09:38.194 INFO:tasks.rados.rados.0.smithi001.stdout:437: rollback oid 26 current snap is 44 2021-01-15T19:09:38.194 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 26 to 36 2021-01-15T19:09:38.195 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 44 2021-01-15T19:09:38.195 INFO:tasks.rados.rados.0.smithi001.stdout:438: seq_num 165 ranges {6324224=368640} 2021-01-15T19:09:38.233 INFO:tasks.rados.rados.0.smithi001.stdout:438: writing smithi00140323-13 from 6324224 to 6692864 tid 1 2021-01-15T19:09:38.234 INFO:tasks.rados.rados.0.smithi001.stdout:439: snap_create 2021-01-15T19:09:38.234 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 154 (ObjNum 135 snap 32 seq_num 135) dirty exists 2021-01-15T19:09:38.234 INFO:tasks.rados.rados.0.smithi001.stdout:437: finishing rollback tid 0 to smithi00140323-26 2021-01-15T19:09:38.235 INFO:tasks.rados.rados.0.smithi001.stdout:437: finishing rollback tid 1 to smithi00140323-26 2021-01-15T19:09:38.235 INFO:tasks.rados.rados.0.smithi001.stdout:437: finishing rollback tid 2 to smithi00140323-26 2021-01-15T19:09:38.235 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 66 (ObjNum 103 snap 17 seq_num 103) dirty exists 2021-01-15T19:09:38.235 INFO:tasks.rados.rados.0.smithi001.stdout:436: finishing copy_from to smithi00140323-43 2021-01-15T19:09:38.236 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 34 (ObjNum 127 snap 30 seq_num 127) dirty exists 2021-01-15T19:09:38.238 INFO:tasks.rados.rados.0.smithi001.stdout:436: finishing copy_from racing read to smithi00140323-43 2021-01-15T19:09:38.250 INFO:tasks.rados.rados.0.smithi001.stdout:438: finishing write tid 1 to smithi00140323-13 2021-01-15T19:09:38.250 INFO:tasks.rados.rados.0.smithi001.stdout:438: finishing write tid 2 to smithi00140323-13 2021-01-15T19:09:38.251 INFO:tasks.rados.rados.0.smithi001.stdout:438: finishing write tid 3 to smithi00140323-13 2021-01-15T19:09:38.251 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 202 (ObjNum 165 snap 44 seq_num 165) dirty exists 2021-01-15T19:09:38.252 INFO:tasks.rados.rados.0.smithi001.stdout:438: left oid 13 (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:09:38.254 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:38.253+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:38.330 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:38.331 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:38.331 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:38.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:38.355 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:38.354+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:38.456 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:38.454+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:38.466 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:38.466 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:38.466 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:38.467 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:38.467 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:38.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:38.520 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:38.564 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:38.563+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:38.641 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:38.641 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.352954627000003, 2021-01-15T19:09:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.421191860999997, 2021-01-15T19:09:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.418903819999997, 2021-01-15T19:09:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.681 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:38.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.416314194999998, 2021-01-15T19:09:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:38.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:38.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.415405917000001, 2021-01-15T19:09:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:38.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:38.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:38.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:38.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:38.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:38.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.446722010000002, 2021-01-15T19:09:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:38.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:38.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:38.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:38.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.446572273000001, 2021-01-15T19:09:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:38.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:38.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:38.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:38.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:38.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:38.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:38.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:38.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:38.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.446420375000002, 2021-01-15T19:09:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.446295062999994, 2021-01-15T19:09:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.425462488000001, 2021-01-15T19:09:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.419169156000002, 2021-01-15T19:09:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.457109643000003, 2021-01-15T19:09:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.394737663000001, 2021-01-15T19:09:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.848 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.848 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.391435162000001, 2021-01-15T19:09:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.390113296000003, 2021-01-15T19:09:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.389743043000003, 2021-01-15T19:09:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.38940865, 2021-01-15T19:09:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:38.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:38.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.924 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.448147390999999, 2021-01-15T19:09:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.925 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:38.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.455784518999998, 2021-01-15T19:09:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.939 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:38.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:38.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:38.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:38.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:38.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:38.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:38.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:38.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:38.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:38.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:38.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:38.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.425790315, 2021-01-15T19:09:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:38.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:38.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:38.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:38.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:38.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:38.971 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:38.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:38.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:38.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:38.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:38.975 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:38.975 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:38.975 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:38.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:38.977 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:38.977 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:38.978 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:38.971+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:38.978 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:39.074 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:39.072+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:39.104 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:39.104 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:39.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:39.175 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:39.173+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:39.239 INFO:tasks.rados.rados.0.smithi001.stdout:435: done (4 left) 2021-01-15T19:09:39.240 INFO:tasks.rados.rados.0.smithi001.stdout:436: done (3 left) 2021-01-15T19:09:39.240 INFO:tasks.rados.rados.0.smithi001.stdout:437: done (2 left) 2021-01-15T19:09:39.240 INFO:tasks.rados.rados.0.smithi001.stdout:438: done (1 left) 2021-01-15T19:09:39.240 INFO:tasks.rados.rados.0.smithi001.stdout:439: done (0 left) 2021-01-15T19:09:39.241 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 45 2021-01-15T19:09:39.241 INFO:tasks.rados.rados.0.smithi001.stdout:440: seq_num 166 ranges {0=385024} 2021-01-15T19:09:39.242 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:39.243 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:39.243 INFO:tasks.rados.rados.0.smithi001.stdout:440: writing smithi00140323-14 from 0 to 385024 tid 1 2021-01-15T19:09:39.244 INFO:tasks.rados.rados.0.smithi001.stdout:441: snap_remove snap 44 2021-01-15T19:09:39.251 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:39.295 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:39.293+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:39.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:09:39.377 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:39.378 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:39.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:39.394 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:39.393+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:39.479 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:39.477+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:39.541 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:39.542 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:39.552 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:39.594 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:39.592+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:39.680 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:39.679+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:39.724 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:39.724 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:39.734 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:39.782 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:39.781+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:39.867 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:39.867 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:39.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:39.921 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:39.920+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:39.983 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:39.981+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:40.008 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:40.009 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:40.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:40.084 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:40.083+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:40.185 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:40.183+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:40.190 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:40.191 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:40.199 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:40.244 INFO:tasks.rados.rados.0.smithi001.stdout:441: done (1 left) 2021-01-15T19:09:40.244 INFO:tasks.rados.rados.0.smithi001.stdout:442: rmattr oid 12 current snap is 45 2021-01-15T19:09:40.245 INFO:tasks.rados.rados.0.smithi001.stdout:443: snap_create 2021-01-15T19:09:40.246 INFO:tasks.rados.rados.0.smithi001.stdout:440: finishing write tid 1 to smithi00140323-14 2021-01-15T19:09:40.246 INFO:tasks.rados.rados.0.smithi001.stdout:440: finishing write tid 2 to smithi00140323-14 2021-01-15T19:09:40.246 INFO:tasks.rados.rados.0.smithi001.stdout:440: finishing write tid 3 to smithi00140323-14 2021-01-15T19:09:40.246 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 81 (ObjNum 166 snap 45 seq_num 166) dirty exists 2021-01-15T19:09:40.247 INFO:tasks.rados.rados.0.smithi001.stdout:440: left oid 14 (ObjNum 166 snap 45 seq_num 166) 2021-01-15T19:09:40.248 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 0 (ObjNum 127 snap 30 seq_num 127) dirty exists 2021-01-15T19:09:40.248 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:09:40.298 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:40.297+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:40.371 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2021-01-15T19:09:40.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:40.372 INFO:tasks.thrashosds.thrasher:fixing pg num pool device_health_metrics 2021-01-15T19:09:40.372 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:09:40.375 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:40.376 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:40.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:40.389 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:40.387+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:40.490 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:40.488+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:40.564 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:40.564 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:40.575 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:40.615 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:40.614+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:40.691 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:40.690+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:40.695 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:09:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:09:40.706 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2021-01-15T19:09:40.238847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173056000,"num_objects":138,"num_object_clones":89,"num_object_copies":411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":8,"num_read":326,"num_read_kb":285912,"num_write":485,"num_write_kb":118914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":113893376,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1957,"ondisk_log_size":1957,"up":125,"acting":125,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":387452,"kb_used_data":265220,"kb_used_omap":0,"kb_used_meta":122176,"kb_avail":660215428,"statfs":{"total":676457349120,"available":676060598272,"internally_reserved":0,"allocated":271585280,"data_stored":363762674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":125108224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":286720,"num_objects":-3,"num_object_clones":-3,"num_object_copies":-9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3,"num_whiteouts":0,"num_read":15,"num_read_kb":21424,"num_write":45,"num_write_kb":2616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024482"},"pg_stats":[{"pgid":"3.19","version":"102'55","reported_seq":"200","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.194646+0000","last_change":"2021-01-15T19:09:27.212298+0000","last_active":"2021-01-15T19:09:34.194646+0000","last_peered":"2021-01-15T19:09:34.194646+0000","last_clean":"2021-01-15T19:09:34.194646+0000","last_became_active":"2021-01-15T19:08:46.196966+0000","last_became_peered":"2021-01-15T19:08:46.196966+0000","last_unstale":"2021-01-15T19:09:34.194646+0000","last_undegraded":"2021-01-15T19:09:34.194646+0000","last_fullsized":"2021-01-15T19:09:34.194646+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'51","last_scrub_stamp":"2021-01-15T19:09:27.212108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:27.212108+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":6,"stat_sum":{"num_bytes":5300224,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.18","version":"96'58","reported_seq":"206","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.194679+0000","last_change":"2021-01-15T19:09:24.220387+0000","last_active":"2021-01-15T19:09:34.194679+0000","last_peered":"2021-01-15T19:09:34.194679+0000","last_clean":"2021-01-15T19:09:22.200101+0000","last_became_active":"2021-01-15T19:09:24.220387+0000","last_became_peered":"2021-01-15T19:09:24.220387+0000","last_unstale":"2021-01-15T19:09:34.194679+0000","last_undegraded":"2021-01-15T19:09:24.208924+0000","last_fullsized":"2021-01-15T19:09:24.208713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"51'34","last_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_deep_scrub":"51'34","last_deep_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.694318+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":8,"stat_sum":{"num_bytes":1523712,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":8,"num_write_kb":2072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2147483647,7],"acting":[4,2147483647,7],"avail_no_missing":["4(0)","7(2)"],"object_location_counts":[{"shards":"4(0),7(2)","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"50'30","reported_seq":"78","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.217388+0000","last_change":"2021-01-15T19:09:36.217954+0000","last_active":"2021-01-15T19:09:36.217388+0000","last_peered":"2021-01-15T19:09:36.217388+0000","last_clean":"2021-01-15T19:09:36.217388+0000","last_became_active":"2021-01-15T19:09:21.207252+0000","last_became_peered":"2021-01-15T19:09:21.207252+0000","last_unstale":"2021-01-15T19:09:36.217388+0000","last_undegraded":"2021-01-15T19:09:36.217388+0000","last_fullsized":"2021-01-15T19:09:36.217388+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'30","last_scrub_stamp":"2021-01-15T19:09:25.976466+0000","last_deep_scrub":"50'30","last_deep_scrub_stamp":"2021-01-15T19:09:04.507336+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.976466+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"3.16","version":"106'65","reported_seq":"261","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.700398+0000","last_change":"2021-01-15T19:09:34.700398+0000","last_active":"2021-01-15T19:09:34.700398+0000","last_peered":"2021-01-15T19:09:34.700398+0000","last_clean":"2021-01-15T19:09:34.700398+0000","last_became_active":"2021-01-15T19:09:15.343356+0000","last_became_peered":"2021-01-15T19:09:15.343356+0000","last_unstale":"2021-01-15T19:09:34.700398+0000","last_undegraded":"2021-01-15T19:09:34.700398+0000","last_fullsized":"2021-01-15T19:09:34.700398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'32","last_scrub_stamp":"2021-01-15T19:08:51.873289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.873289+0000","log_size":65,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5185536,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":10,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2621440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]},{"pgid":"3.15","version":"108'52","reported_seq":"187","reported_epoch":"108","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:36.238215+0000","last_change":"2021-01-15T19:09:24.242617+0000","last_active":"2021-01-15T19:09:36.238215+0000","last_peered":"2021-01-15T19:09:36.238215+0000","last_clean":"2021-01-15T19:09:22.196695+0000","last_became_active":"2021-01-15T19:09:24.242617+0000","last_became_peered":"2021-01-15T19:09:24.242617+0000","last_unstale":"2021-01-15T19:09:36.238215+0000","last_undegraded":"2021-01-15T19:09:24.209123+0000","last_fullsized":"2021-01-15T19:09:24.208870+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'32","last_scrub_stamp":"2021-01-15T19:09:03.911966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:03.911966+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"stat_sum":{"num_bytes":2670592,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2147483647,3],"acting":[0,2147483647,3],"avail_no_missing":["0(0)","3(2)"],"object_location_counts":[{"shards":"0(0),3(2)","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"108'67","reported_seq":"113","reported_epoch":"108","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:36.265395+0000","last_change":"2021-01-15T19:09:24.226004+0000","last_active":"2021-01-15T19:09:36.265395+0000","last_peered":"2021-01-15T19:09:36.265395+0000","last_clean":"2021-01-15T19:09:22.213124+0000","last_became_active":"2021-01-15T19:09:24.226004+0000","last_became_peered":"2021-01-15T19:09:24.226004+0000","last_unstale":"2021-01-15T19:09:36.265395+0000","last_undegraded":"2021-01-15T19:09:24.209319+0000","last_fullsized":"2021-01-15T19:09:24.208964+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":67,"ondisk_log_size":67,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"stat_sum":{"num_bytes":6938624,"num_objects":8,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":10,"num_read_kb":4024,"num_write":17,"num_write_kb":6776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":10870784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,2147483647],"acting":[0,2,2147483647],"avail_no_missing":["0(0)","2(1)"],"object_location_counts":[{"shards":"0(0),2(1)","objects":8}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"194","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.260293+0000","last_change":"2021-01-15T19:09:36.260747+0000","last_active":"2021-01-15T19:09:36.260293+0000","last_peered":"2021-01-15T19:09:36.260293+0000","last_clean":"2021-01-15T19:09:36.260293+0000","last_became_active":"2021-01-15T19:08:43.203849+0000","last_became_peered":"2021-01-15T19:08:43.203849+0000","last_unstale":"2021-01-15T19:09:36.260293+0000","last_undegraded":"2021-01-15T19:09:36.260293+0000","last_fullsized":"2021-01-15T19:09:36.260293+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.175599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"3.12","version":"97'74","reported_seq":"303","reported_epoch":"107","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:35.788067+0000","last_change":"2021-01-15T19:09:24.233757+0000","last_active":"2021-01-15T19:09:35.788067+0000","last_peered":"2021-01-15T19:09:35.788067+0000","last_clean":"2021-01-15T19:09:21.719663+0000","last_became_active":"2021-01-15T19:09:24.233757+0000","last_became_peered":"2021-01-15T19:09:24.233757+0000","last_unstale":"2021-01-15T19:09:35.788067+0000","last_undegraded":"2021-01-15T19:09:24.210370+0000","last_fullsized":"2021-01-15T19:09:24.209002+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'42","last_scrub_stamp":"2021-01-15T19:09:02.894120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.894120+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"stat_sum":{"num_bytes":14270464,"num_objects":9,"num_object_clones":6,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":10,"num_read_kb":16275,"num_write":23,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":34463744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,0,7],"acting":[2147483647,0,7],"avail_no_missing":["0(1)","7(2)"],"object_location_counts":[{"shards":"0(1),7(2)","objects":9}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"14'2","reported_seq":"61","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:35.215279+0000","last_change":"2021-01-15T19:08:53.068011+0000","last_active":"2021-01-15T19:09:35.215279+0000","last_peered":"2021-01-15T19:09:35.215279+0000","last_clean":"2021-01-15T19:09:35.215279+0000","last_became_active":"2021-01-15T19:08:52.197215+0000","last_became_peered":"2021-01-15T19:08:52.197215+0000","last_unstale":"2021-01-15T19:09:35.215279+0000","last_undegraded":"2021-01-15T19:09:35.215279+0000","last_fullsized":"2021-01-15T19:09:35.215279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"106'89","reported_seq":"349","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.745724+0000","last_change":"2021-01-15T19:09:34.745724+0000","last_active":"2021-01-15T19:09:34.745724+0000","last_peered":"2021-01-15T19:09:34.745724+0000","last_clean":"2021-01-15T19:09:34.745724+0000","last_became_active":"2021-01-15T19:08:41.227163+0000","last_became_peered":"2021-01-15T19:08:41.227163+0000","last_unstale":"2021-01-15T19:09:34.745724+0000","last_undegraded":"2021-01-15T19:09:34.745724+0000","last_fullsized":"2021-01-15T19:09:34.745724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'62","last_scrub_stamp":"2021-01-15T19:09:14.332675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:14.332675+0000","log_size":89,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7774208,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":"79","reported_epoch":"107","state":"active+undersized","last_fresh":"2021-01-15T19:09:35.788529+0000","last_change":"2021-01-15T19:09:24.236652+0000","last_active":"2021-01-15T19:09:35.788529+0000","last_peered":"2021-01-15T19:09:35.788529+0000","last_clean":"2021-01-15T19:09:22.196912+0000","last_became_active":"2021-01-15T19:09:24.236652+0000","last_became_peered":"2021-01-15T19:09:24.236652+0000","last_unstale":"2021-01-15T19:09:35.788529+0000","last_undegraded":"2021-01-15T19:09:35.788529+0000","last_fullsized":"2021-01-15T19:09:24.219472+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.960545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.960545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"64'3","reported_seq":"78","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.193759+0000","last_change":"2021-01-15T19:09:24.216410+0000","last_active":"2021-01-15T19:09:34.193759+0000","last_peered":"2021-01-15T19:09:34.193759+0000","last_clean":"2021-01-15T19:09:22.194961+0000","last_became_active":"2021-01-15T19:09:24.216410+0000","last_became_peered":"2021-01-15T19:09:24.216410+0000","last_unstale":"2021-01-15T19:09:34.193759+0000","last_undegraded":"2021-01-15T19:09:24.204882+0000","last_fullsized":"2021-01-15T19:09:24.204793+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.906630+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"110","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.194169+0000","last_change":"2021-01-15T19:08:38.030592+0000","last_active":"2021-01-15T19:09:34.194169+0000","last_peered":"2021-01-15T19:09:34.194169+0000","last_clean":"2021-01-15T19:09:34.194169+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:09:34.194169+0000","last_undegraded":"2021-01-15T19:09:34.194169+0000","last_fullsized":"2021-01-15T19:09:34.194169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.030523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.030523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"90'48","reported_seq":"290","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.194295+0000","last_change":"2021-01-15T19:09:24.216636+0000","last_active":"2021-01-15T19:09:34.194295+0000","last_peered":"2021-01-15T19:09:34.194295+0000","last_clean":"2021-01-15T19:09:22.200264+0000","last_became_active":"2021-01-15T19:09:24.216636+0000","last_became_peered":"2021-01-15T19:09:24.216636+0000","last_unstale":"2021-01-15T19:09:34.194295+0000","last_undegraded":"2021-01-15T19:09:24.209307+0000","last_fullsized":"2021-01-15T19:09:24.209128+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'37","last_scrub_stamp":"2021-01-15T19:08:54.947352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:54.947352+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":8,"stat_sum":{"num_bytes":4710400,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":13,"num_read_kb":36507,"num_write":2,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2147483647,1],"acting":[4,2147483647,1],"avail_no_missing":["4(0)","1(2)"],"object_location_counts":[{"shards":"1(2),4(0)","objects":3}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"14'2","reported_seq":"28","reported_epoch":"107","state":"active+undersized","last_fresh":"2021-01-15T19:09:35.215534+0000","last_change":"2021-01-15T19:09:24.206432+0000","last_active":"2021-01-15T19:09:35.215534+0000","last_peered":"2021-01-15T19:09:35.215534+0000","last_clean":"2021-01-15T19:08:47.197265+0000","last_became_active":"2021-01-15T19:09:24.206432+0000","last_became_peered":"2021-01-15T19:09:24.206432+0000","last_unstale":"2021-01-15T19:09:35.215534+0000","last_undegraded":"2021-01-15T19:09:35.215534+0000","last_fullsized":"2021-01-15T19:09:24.205246+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:15.692639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:15.692639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"93'55","reported_seq":"317","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.205156+0000","last_change":"2021-01-15T19:09:24.214351+0000","last_active":"2021-01-15T19:09:34.205156+0000","last_peered":"2021-01-15T19:09:34.205156+0000","last_clean":"2021-01-15T19:09:21.214000+0000","last_became_active":"2021-01-15T19:09:24.214351+0000","last_became_peered":"2021-01-15T19:09:24.214351+0000","last_unstale":"2021-01-15T19:09:34.205156+0000","last_undegraded":"2021-01-15T19:09:24.206008+0000","last_fullsized":"2021-01-15T19:09:24.205802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'35","last_scrub_stamp":"2021-01-15T19:08:51.059372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.059372+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":8,"stat_sum":{"num_bytes":2686976,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1825,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,1,4],"acting":[2147483647,1,4],"avail_no_missing":["1(1)","4(2)"],"object_location_counts":[{"shards":"1(1),4(2)","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"108","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:36.017577+0000","last_change":"2021-01-15T19:08:55.065884+0000","last_active":"2021-01-15T19:09:36.017577+0000","last_peered":"2021-01-15T19:09:36.017577+0000","last_clean":"2021-01-15T19:09:36.017577+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:09:36.017577+0000","last_undegraded":"2021-01-15T19:09:36.017577+0000","last_fullsized":"2021-01-15T19:09:36.017577+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_clean_scrub_stamp":"2021-01-15T19:08:55.065810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"124","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.195104+0000","last_change":"2021-01-15T19:09:28.194990+0000","last_active":"2021-01-15T19:09:34.195104+0000","last_peered":"2021-01-15T19:09:34.195104+0000","last_clean":"2021-01-15T19:09:34.195104+0000","last_became_active":"2021-01-15T19:08:52.202463+0000","last_became_peered":"2021-01-15T19:08:52.202463+0000","last_unstale":"2021-01-15T19:09:34.195104+0000","last_undegraded":"2021-01-15T19:09:34.195104+0000","last_fullsized":"2021-01-15T19:09:34.195104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:28.194883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.194883+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"90'79","reported_seq":"302","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.193543+0000","last_change":"2021-01-15T19:09:24.223467+0000","last_active":"2021-01-15T19:09:34.193543+0000","last_peered":"2021-01-15T19:09:34.193543+0000","last_clean":"2021-01-15T19:09:18.213719+0000","last_became_active":"2021-01-15T19:09:24.223467+0000","last_became_peered":"2021-01-15T19:09:24.223467+0000","last_unstale":"2021-01-15T19:09:34.193543+0000","last_undegraded":"2021-01-15T19:09:24.203592+0000","last_fullsized":"2021-01-15T19:09:24.203388+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'47","last_scrub_stamp":"2021-01-15T19:08:53.015941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.015941+0000","log_size":79,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":8,"stat_sum":{"num_bytes":5054464,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":3816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,4],"acting":[2147483647,2,4],"avail_no_missing":["2(1)","4(2)"],"object_location_counts":[{"shards":"2(1),4(2)","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"121","reported_epoch":"107","state":"active+undersized","last_fresh":"2021-01-15T19:09:36.017435+0000","last_change":"2021-01-15T19:09:24.211262+0000","last_active":"2021-01-15T19:09:36.017435+0000","last_peered":"2021-01-15T19:09:36.017435+0000","last_clean":"2021-01-15T19:09:22.484323+0000","last_became_active":"2021-01-15T19:09:24.211262+0000","last_became_peered":"2021-01-15T19:09:24.211262+0000","last_unstale":"2021-01-15T19:09:36.017435+0000","last_undegraded":"2021-01-15T19:09:36.017435+0000","last_fullsized":"2021-01-15T19:09:24.208545+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:43.003765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:43.003765+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"281","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.702609+0000","last_change":"2021-01-15T19:09:34.702609+0000","last_active":"2021-01-15T19:09:34.702609+0000","last_peered":"2021-01-15T19:09:34.702609+0000","last_clean":"2021-01-15T19:09:34.702609+0000","last_became_active":"2021-01-15T19:08:41.234856+0000","last_became_peered":"2021-01-15T19:08:41.234856+0000","last_unstale":"2021-01-15T19:09:34.702609+0000","last_undegraded":"2021-01-15T19:09:34.702609+0000","last_fullsized":"2021-01-15T19:09:34.702609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'18","last_scrub_stamp":"2021-01-15T19:09:16.308952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.308952+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]},{"pgid":"1.0","version":"14'2","reported_seq":"131","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.202480+0000","last_change":"2021-01-15T19:09:21.202209+0000","last_active":"2021-01-15T19:09:34.202480+0000","last_peered":"2021-01-15T19:09:34.202480+0000","last_clean":"2021-01-15T19:09:34.202480+0000","last_became_active":"2021-01-15T19:09:21.201680+0000","last_became_peered":"2021-01-15T19:09:21.201680+0000","last_unstale":"2021-01-15T19:09:34.202480+0000","last_undegraded":"2021-01-15T19:09:34.202480+0000","last_fullsized":"2021-01-15T19:09:34.202480+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:52.730150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.730150+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"108","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:36.017448+0000","last_change":"2021-01-15T19:09:04.092968+0000","last_active":"2021-01-15T19:09:36.017448+0000","last_peered":"2021-01-15T19:09:36.017448+0000","last_clean":"2021-01-15T19:09:36.017448+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:09:36.017448+0000","last_undegraded":"2021-01-15T19:09:36.017448+0000","last_fullsized":"2021-01-15T19:09:36.017448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:04.092902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:04.092902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"67","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:35.788323+0000","last_change":"2021-01-15T19:09:16.945459+0000","last_active":"2021-01-15T19:09:35.788323+0000","last_peered":"2021-01-15T19:09:35.788323+0000","last_clean":"2021-01-15T19:09:35.788323+0000","last_became_active":"2021-01-15T19:08:52.205001+0000","last_became_peered":"2021-01-15T19:08:52.205001+0000","last_unstale":"2021-01-15T19:09:35.788323+0000","last_undegraded":"2021-01-15T19:09:35.788323+0000","last_fullsized":"2021-01-15T19:09:35.788323+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:16.945387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.945387+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"106'70","reported_seq":"315","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.702181+0000","last_change":"2021-01-15T19:09:34.702181+0000","last_active":"2021-01-15T19:09:34.702181+0000","last_peered":"2021-01-15T19:09:34.702181+0000","last_clean":"2021-01-15T19:09:34.702181+0000","last_became_active":"2021-01-15T19:09:21.202949+0000","last_became_peered":"2021-01-15T19:09:21.202949+0000","last_unstale":"2021-01-15T19:09:34.702181+0000","last_undegraded":"2021-01-15T19:09:34.702181+0000","last_fullsized":"2021-01-15T19:09:34.702181+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'68","last_scrub_stamp":"2021-01-15T19:09:29.177377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.177377+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2703360,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"127","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:35.215398+0000","last_change":"2021-01-15T19:09:18.204776+0000","last_active":"2021-01-15T19:09:35.215398+0000","last_peered":"2021-01-15T19:09:35.215398+0000","last_clean":"2021-01-15T19:09:35.215398+0000","last_became_active":"2021-01-15T19:09:18.204537+0000","last_became_peered":"2021-01-15T19:09:18.204537+0000","last_unstale":"2021-01-15T19:09:35.215398+0000","last_undegraded":"2021-01-15T19:09:35.215398+0000","last_fullsized":"2021-01-15T19:09:35.215398+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:38.923498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:38.923498+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"27","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.194810+0000","last_change":"2021-01-15T19:09:24.216624+0000","last_active":"2021-01-15T19:09:34.194810+0000","last_peered":"2021-01-15T19:09:34.194810+0000","last_clean":"2021-01-15T19:08:47.193108+0000","last_became_active":"2021-01-15T19:09:24.216624+0000","last_became_peered":"2021-01-15T19:09:24.216624+0000","last_unstale":"2021-01-15T19:09:34.194810+0000","last_undegraded":"2021-01-15T19:09:24.212416+0000","last_fullsized":"2021-01-15T19:09:24.212321+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:07:48.500993+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"260","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.241236+0000","last_change":"2021-01-15T19:09:36.241839+0000","last_active":"2021-01-15T19:09:36.241236+0000","last_peered":"2021-01-15T19:09:36.241236+0000","last_clean":"2021-01-15T19:09:36.241236+0000","last_became_active":"2021-01-15T19:08:41.228264+0000","last_became_peered":"2021-01-15T19:08:41.228264+0000","last_unstale":"2021-01-15T19:09:36.241236+0000","last_undegraded":"2021-01-15T19:09:36.241236+0000","last_fullsized":"2021-01-15T19:09:36.241236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:52.822027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.822027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"57","reported_epoch":"106","state":"active+undersized","last_fresh":"2021-01-15T19:09:34.202385+0000","last_change":"2021-01-15T19:09:24.212158+0000","last_active":"2021-01-15T19:09:34.202385+0000","last_peered":"2021-01-15T19:09:34.202385+0000","last_clean":"2021-01-15T19:08:33.956428+0000","last_became_active":"2021-01-15T19:09:24.212158+0000","last_became_peered":"2021-01-15T19:09:24.212158+0000","last_unstale":"2021-01-15T19:09:34.202385+0000","last_undegraded":"2021-01-15T19:09:34.202385+0000","last_fullsized":"2021-01-15T19:09:24.205838+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:14.728250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:14.728250+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"135","reported_epoch":"106","state":"active+undersized","last_fresh":"2021-01-15T19:09:34.205039+0000","last_change":"2021-01-15T19:09:24.212235+0000","last_active":"2021-01-15T19:09:34.205039+0000","last_peered":"2021-01-15T19:09:34.205039+0000","last_clean":"2021-01-15T19:09:22.199740+0000","last_became_active":"2021-01-15T19:09:24.212235+0000","last_became_peered":"2021-01-15T19:09:24.212235+0000","last_unstale":"2021-01-15T19:09:34.205039+0000","last_undegraded":"2021-01-15T19:09:34.205039+0000","last_fullsized":"2021-01-15T19:09:24.206594+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"106'70","reported_seq":"309","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.227697+0000","last_change":"2021-01-15T19:09:24.221055+0000","last_active":"2021-01-15T19:09:34.227697+0000","last_peered":"2021-01-15T19:09:34.227697+0000","last_clean":"2021-01-15T19:09:22.194819+0000","last_became_active":"2021-01-15T19:09:24.221055+0000","last_became_peered":"2021-01-15T19:09:24.221055+0000","last_unstale":"2021-01-15T19:09:34.227697+0000","last_undegraded":"2021-01-15T19:09:24.204503+0000","last_fullsized":"2021-01-15T19:09:24.204342+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'31","last_scrub_stamp":"2021-01-15T19:08:49.815451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:49.815451+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":8,"stat_sum":{"num_bytes":8306688,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":8643,"num_write":9,"num_write_kb":5784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,2147483647,4],"acting":[2,2147483647,4],"avail_no_missing":["2(0)","4(2)"],"object_location_counts":[{"shards":"2(0),4(2)","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"58","reported_epoch":"106","state":"active+undersized","last_fresh":"2021-01-15T19:09:34.194154+0000","last_change":"2021-01-15T19:09:24.216705+0000","last_active":"2021-01-15T19:09:34.194154+0000","last_peered":"2021-01-15T19:09:34.194154+0000","last_clean":"2021-01-15T19:08:33.194477+0000","last_became_active":"2021-01-15T19:09:24.216705+0000","last_became_peered":"2021-01-15T19:09:24.216705+0000","last_unstale":"2021-01-15T19:09:34.194154+0000","last_undegraded":"2021-01-15T19:09:34.194154+0000","last_fullsized":"2021-01-15T19:09:24.204328+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:39.732565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:39.732565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.a","version":"14'2","reported_seq":"135","reported_epoch":"106","state":"active+undersized","last_fresh":"2021-01-15T19:09:34.202581+0000","last_change":"2021-01-15T19:09:24.212458+0000","last_active":"2021-01-15T19:09:34.202581+0000","last_peered":"2021-01-15T19:09:34.202581+0000","last_clean":"2021-01-15T19:09:22.199641+0000","last_became_active":"2021-01-15T19:09:24.212458+0000","last_became_peered":"2021-01-15T19:09:24.212458+0000","last_unstale":"2021-01-15T19:09:34.202581+0000","last_undegraded":"2021-01-15T19:09:34.202581+0000","last_fullsized":"2021-01-15T19:09:24.206163+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:01.619232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:01.619232+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"106'189","reported_seq":"434","reported_epoch":"106","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:09:35.212501+0000","last_change":"2021-01-15T19:09:34.702683+0000","last_active":"2021-01-15T19:09:35.212501+0000","last_peered":"2021-01-15T19:09:35.212501+0000","last_clean":"2021-01-15T19:09:35.212501+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:09:35.212501+0000","last_undegraded":"2021-01-15T19:09:35.212501+0000","last_fullsized":"2021-01-15T19:09:35.212501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'109","last_scrub_stamp":"2021-01-15T19:09:15.296470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:15.296470+0000","log_size":189,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":17301504,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":29,"num_write_kb":7112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":"71","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.194451+0000","last_change":"2021-01-15T19:08:50.981704+0000","last_active":"2021-01-15T19:09:34.194451+0000","last_peered":"2021-01-15T19:09:34.194451+0000","last_clean":"2021-01-15T19:09:34.194451+0000","last_became_active":"2021-01-15T19:08:37.215228+0000","last_became_peered":"2021-01-15T19:08:37.215228+0000","last_unstale":"2021-01-15T19:09:34.194451+0000","last_undegraded":"2021-01-15T19:09:34.194451+0000","last_fullsized":"2021-01-15T19:09:34.194451+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:50.981612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:50.981612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"108'111","reported_seq":"394","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.217398+0000","last_change":"2021-01-15T19:09:36.217398+0000","last_active":"2021-01-15T19:09:36.217398+0000","last_peered":"2021-01-15T19:09:36.217398+0000","last_clean":"2021-01-15T19:09:36.217398+0000","last_became_active":"2021-01-15T19:09:15.341093+0000","last_became_peered":"2021-01-15T19:09:15.341093+0000","last_unstale":"2021-01-15T19:09:36.217398+0000","last_undegraded":"2021-01-15T19:09:36.217398+0000","last_fullsized":"2021-01-15T19:09:36.217398+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'61","last_scrub_stamp":"2021-01-15T19:09:07.109680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:07.109680+0000","log_size":111,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10256384,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":9,"num_read_kb":11499,"num_write":27,"num_write_kb":2336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"72","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:35.787968+0000","last_change":"2021-01-15T19:08:51.826450+0000","last_active":"2021-01-15T19:09:35.787968+0000","last_peered":"2021-01-15T19:09:35.787968+0000","last_clean":"2021-01-15T19:09:35.787968+0000","last_became_active":"2021-01-15T19:08:37.216976+0000","last_became_peered":"2021-01-15T19:08:37.216976+0000","last_unstale":"2021-01-15T19:09:35.787968+0000","last_undegraded":"2021-01-15T19:09:35.787968+0000","last_fullsized":"2021-01-15T19:09:35.787968+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:51.826360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.826360+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"14'2","reported_seq":"130","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.202370+0000","last_change":"2021-01-15T19:09:06.583849+0000","last_active":"2021-01-15T19:09:34.202370+0000","last_peered":"2021-01-15T19:09:34.202370+0000","last_clean":"2021-01-15T19:09:34.202370+0000","last_became_active":"2021-01-15T19:08:52.204479+0000","last_became_peered":"2021-01-15T19:08:52.204479+0000","last_unstale":"2021-01-15T19:09:34.202370+0000","last_undegraded":"2021-01-15T19:09:34.202370+0000","last_fullsized":"2021-01-15T19:09:34.202370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:06.583774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:06.583774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"106'132","reported_seq":"398","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.263953+0000","last_change":"2021-01-15T19:09:36.263953+0000","last_active":"2021-01-15T19:09:36.263953+0000","last_peered":"2021-01-15T19:09:36.263953+0000","last_clean":"2021-01-15T19:09:36.263953+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:09:36.263953+0000","last_undegraded":"2021-01-15T19:09:36.263953+0000","last_fullsized":"2021-01-15T19:09:36.263953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'120","last_scrub_stamp":"2021-01-15T19:09:30.324786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.324786+0000","log_size":132,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8306688,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":46,"num_read_kb":43721,"num_write":62,"num_write_kb":14288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.b","version":"0'0","reported_seq":"107","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:36.017570+0000","last_change":"2021-01-15T19:09:05.065723+0000","last_active":"2021-01-15T19:09:36.017570+0000","last_peered":"2021-01-15T19:09:36.017570+0000","last_clean":"2021-01-15T19:09:36.017570+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:09:36.017570+0000","last_undegraded":"2021-01-15T19:09:36.017570+0000","last_fullsized":"2021-01-15T19:09:36.017570+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:05.065643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:05.065643+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"14'2","reported_seq":"134","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.202530+0000","last_change":"2021-01-15T19:09:18.403315+0000","last_active":"2021-01-15T19:09:34.202530+0000","last_peered":"2021-01-15T19:09:34.202530+0000","last_clean":"2021-01-15T19:09:34.202530+0000","last_became_active":"2021-01-15T19:08:52.203041+0000","last_became_peered":"2021-01-15T19:08:52.203041+0000","last_unstale":"2021-01-15T19:09:34.202530+0000","last_undegraded":"2021-01-15T19:09:34.202530+0000","last_fullsized":"2021-01-15T19:09:34.202530+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:18.403239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.403239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"108'36","reported_seq":"279","reported_epoch":"108","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:36.242834+0000","last_change":"2021-01-15T19:09:24.215910+0000","last_active":"2021-01-15T19:09:36.242834+0000","last_peered":"2021-01-15T19:09:36.242834+0000","last_clean":"2021-01-15T19:09:22.195163+0000","last_became_active":"2021-01-15T19:09:24.215910+0000","last_became_peered":"2021-01-15T19:09:24.215910+0000","last_unstale":"2021-01-15T19:09:36.242834+0000","last_undegraded":"2021-01-15T19:09:24.205508+0000","last_fullsized":"2021-01-15T19:09:24.205398+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"stat_sum":{"num_bytes":5758976,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":33972,"num_write":14,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2147483647,0],"acting":[7,2147483647,0],"avail_no_missing":["7(0)","0(2)"],"object_location_counts":[{"shards":"0(2),7(0)","objects":6}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":"110","reported_epoch":"107","state":"active+undersized","last_fresh":"2021-01-15T19:09:35.788064+0000","last_change":"2021-01-15T19:09:24.229488+0000","last_active":"2021-01-15T19:09:35.788064+0000","last_peered":"2021-01-15T19:09:35.788064+0000","last_clean":"2021-01-15T19:09:16.350211+0000","last_became_active":"2021-01-15T19:09:24.229488+0000","last_became_peered":"2021-01-15T19:09:24.229488+0000","last_unstale":"2021-01-15T19:09:35.788064+0000","last_undegraded":"2021-01-15T19:09:35.788064+0000","last_fullsized":"2021-01-15T19:09:24.218132+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:40.178185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:40.178185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"91'152","reported_seq":"461","reported_epoch":"108","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:36.238344+0000","last_change":"2021-01-15T19:09:24.217708+0000","last_active":"2021-01-15T19:09:36.238344+0000","last_peered":"2021-01-15T19:09:36.238344+0000","last_clean":"2021-01-15T19:09:22.207196+0000","last_became_active":"2021-01-15T19:09:24.217708+0000","last_became_peered":"2021-01-15T19:09:24.217708+0000","last_unstale":"2021-01-15T19:09:36.238344+0000","last_undegraded":"2021-01-15T19:09:24.208721+0000","last_fullsized":"2021-01-15T19:09:24.208496+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":152,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":9,"stat_sum":{"num_bytes":27254784,"num_objects":16,"num_object_clones":11,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":67,"num_read_kb":77609,"num_write":70,"num_write_kb":27408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":37576704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2147483647],"acting":[0,7,2147483647],"avail_no_missing":["0(0)","7(1)"],"object_location_counts":[{"shards":"0(0),7(1)","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":"33","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.204975+0000","last_change":"2021-01-15T19:09:25.416216+0000","last_active":"2021-01-15T19:09:34.204975+0000","last_peered":"2021-01-15T19:09:34.204975+0000","last_clean":"2021-01-15T19:09:34.204975+0000","last_became_active":"2021-01-15T19:09:18.191906+0000","last_became_peered":"2021-01-15T19:09:18.191906+0000","last_unstale":"2021-01-15T19:09:34.204975+0000","last_undegraded":"2021-01-15T19:09:34.204975+0000","last_fullsized":"2021-01-15T19:09:34.204975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:25.416133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.416133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"108'112","reported_seq":"385","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.261876+0000","last_change":"2021-01-15T19:09:36.239112+0000","last_active":"2021-01-15T19:09:36.261876+0000","last_peered":"2021-01-15T19:09:36.261876+0000","last_clean":"2021-01-15T19:09:36.261876+0000","last_became_active":"2021-01-15T19:09:15.345068+0000","last_became_peered":"2021-01-15T19:09:15.345068+0000","last_unstale":"2021-01-15T19:09:36.261876+0000","last_undegraded":"2021-01-15T19:09:36.261876+0000","last_fullsized":"2021-01-15T19:09:36.261876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'91","last_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_deep_scrub":"101'91","last_deep_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.016550+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6217728,"num_objects":9,"num_object_clones":6,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":41,"num_read_kb":11346,"num_write":73,"num_write_kb":7760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"58","reported_epoch":"107","state":"active+undersized","last_fresh":"2021-01-15T19:09:35.788464+0000","last_change":"2021-01-15T19:09:24.244802+0000","last_active":"2021-01-15T19:09:35.788464+0000","last_peered":"2021-01-15T19:09:35.788464+0000","last_clean":"2021-01-15T19:08:33.194477+0000","last_became_active":"2021-01-15T19:09:24.244802+0000","last_became_peered":"2021-01-15T19:09:24.244802+0000","last_unstale":"2021-01-15T19:09:35.788464+0000","last_undegraded":"2021-01-15T19:09:35.788464+0000","last_fullsized":"2021-01-15T19:09:24.210736+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:50.105418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:50.105418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"106'73","reported_seq":"354","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.704907+0000","last_change":"2021-01-15T19:09:34.704907+0000","last_active":"2021-01-15T19:09:34.704907+0000","last_peered":"2021-01-15T19:09:34.704907+0000","last_clean":"2021-01-15T19:09:34.704907+0000","last_became_active":"2021-01-15T19:09:21.208779+0000","last_became_peered":"2021-01-15T19:09:21.208779+0000","last_unstale":"2021-01-15T19:09:34.704907+0000","last_undegraded":"2021-01-15T19:09:34.704907+0000","last_fullsized":"2021-01-15T19:09:34.704907+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'59","last_scrub_stamp":"2021-01-15T19:09:28.771226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.771226+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6774784,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":559,"num_write":20,"num_write_kb":6800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]},{"pgid":"2.f","version":"0'0","reported_seq":"116","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:35.788403+0000","last_change":"2021-01-15T19:09:18.202944+0000","last_active":"2021-01-15T19:09:35.788403+0000","last_peered":"2021-01-15T19:09:35.788403+0000","last_clean":"2021-01-15T19:09:35.788403+0000","last_became_active":"2021-01-15T19:09:18.202311+0000","last_became_peered":"2021-01-15T19:09:18.202311+0000","last_unstale":"2021-01-15T19:09:35.788403+0000","last_undegraded":"2021-01-15T19:09:35.788403+0000","last_fullsized":"2021-01-15T19:09:35.788403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"106'111","reported_seq":"358","reported_epoch":"108","state":"active+clean","last_fresh":"2021-01-15T19:09:36.238624+0000","last_change":"2021-01-15T19:09:36.239057+0000","last_active":"2021-01-15T19:09:36.238624+0000","last_peered":"2021-01-15T19:09:36.238624+0000","last_clean":"2021-01-15T19:09:36.238624+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:09:36.238624+0000","last_undegraded":"2021-01-15T19:09:36.238624+0000","last_fullsized":"2021-01-15T19:09:36.238624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'93","last_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_deep_scrub":"100'93","last_deep_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.069708+0000","log_size":111,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8151040,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":20,"num_read_kb":2624,"num_write":44,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.e","version":"0'0","reported_seq":"125","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.194272+0000","last_change":"2021-01-15T19:09:18.205520+0000","last_active":"2021-01-15T19:09:34.194272+0000","last_peered":"2021-01-15T19:09:34.194272+0000","last_clean":"2021-01-15T19:09:34.194272+0000","last_became_active":"2021-01-15T19:09:18.205134+0000","last_became_peered":"2021-01-15T19:09:18.205134+0000","last_unstale":"2021-01-15T19:09:34.194272+0000","last_undegraded":"2021-01-15T19:09:34.194272+0000","last_fullsized":"2021-01-15T19:09:34.194272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.084516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.084516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"91'58","reported_seq":"239","reported_epoch":"106","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:09:34.193291+0000","last_change":"2021-01-15T19:09:24.217237+0000","last_active":"2021-01-15T19:09:34.193291+0000","last_peered":"2021-01-15T19:09:34.193291+0000","last_clean":"2021-01-15T19:09:19.251636+0000","last_became_active":"2021-01-15T19:09:24.217237+0000","last_became_peered":"2021-01-15T19:09:24.217237+0000","last_unstale":"2021-01-15T19:09:34.193291+0000","last_undegraded":"2021-01-15T19:09:24.203079+0000","last_fullsized":"2021-01-15T19:09:24.202931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":8,"stat_sum":{"num_bytes":8798208,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":11,"num_read_kb":13680,"num_write":26,"num_write_kb":8416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,1],"acting":[2147483647,2,1],"avail_no_missing":["2(1)","1(2)"],"object_location_counts":[{"shards":"1(2),2(1)","objects":4}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":"71","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:34.194673+0000","last_change":"2021-01-15T19:08:42.076013+0000","last_active":"2021-01-15T19:09:34.194673+0000","last_peered":"2021-01-15T19:09:34.194673+0000","last_clean":"2021-01-15T19:09:34.194673+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:09:34.194673+0000","last_undegraded":"2021-01-15T19:09:34.194673+0000","last_fullsized":"2021-01-15T19:09:34.194673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:08:42.075929+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"80","reported_epoch":"107","state":"active+clean","last_fresh":"2021-01-15T19:09:35.787953+0000","last_change":"2021-01-15T19:09:12.353642+0000","last_active":"2021-01-15T19:09:35.787953+0000","last_peered":"2021-01-15T19:09:35.787953+0000","last_clean":"2021-01-15T19:09:35.787953+0000","last_became_active":"2021-01-15T19:09:12.353214+0000","last_became_peered":"2021-01-15T19:09:12.353214+0000","last_unstale":"2021-01-15T19:09:35.787953+0000","last_undegraded":"2021-01-15T19:09:35.787953+0000","last_fullsized":"2021-01-15T19:09:35.787953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.902685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.902685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"106'130","reported_seq":"379","reported_epoch":"106","state":"active+clean","last_fresh":"2021-01-15T19:09:35.215816+0000","last_change":"2021-01-15T19:09:35.215816+0000","last_active":"2021-01-15T19:09:35.215816+0000","last_peered":"2021-01-15T19:09:35.215816+0000","last_clean":"2021-01-15T19:09:35.215816+0000","last_became_active":"2021-01-15T19:09:21.202752+0000","last_became_peered":"2021-01-15T19:09:21.202752+0000","last_unstale":"2021-01-15T19:09:35.215816+0000","last_undegraded":"2021-01-15T19:09:35.215816+0000","last_fullsized":"2021-01-15T19:09:35.215816+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'67","last_scrub_stamp":"2021-01-15T19:08:49.729161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:49.729161+0000","log_size":130,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7110656,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":22,"num_read_kb":23622,"num_write":8,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":12124160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19","length":"1"},{"start":"1b","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3e","length":"1"},{"start":"43","length":"1"},{"start":"46","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":173056000,"num_objects":135,"num_object_clones":89,"num_object_copies":405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":8,"num_read":326,"num_read_kb":285912,"num_write":484,"num_write_kb":118912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":113893376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282120192,"data_stored":386318336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1934,"ondisk_log_size":1934,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":17,"acting":17,"num_store_stats":8},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70788,"kb_used_data":54076,"kb_used_omap":0,"kb_used_meta":16704,"kb_avail":94301052,"statfs":{"total":96636764160,"available":96564277248,"internally_reserved":0,"allocated":55373824,"data_stored":79312536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17104896},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.044},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.1519999999999999}]},{"osd":1,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.079}]},{"osd":2,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0189999999999999}]},{"osd":3,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.006}]},{"osd":4,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.78500000000000003}]},{"osd":5,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.88}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64740,"kb_used_data":49116,"kb_used_omap":0,"kb_used_meta":15616,"kb_avail":94307100,"statfs":{"total":96636764160,"available":96570470400,"internally_reserved":0,"allocated":50294784,"data_stored":57212888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15990784},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.272}]},{"osd":1,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.052}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.0860000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":3.2930000000000001,"5min":3.2930000000000001,"15min":3.2930000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.1539999999999999}]},{"osd":4,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":3.5870000000000002,"5min":3.5870000000000002,"15min":3.5870000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.3250000000000002,"5min":3.3250000000000002,"15min":3.3250000000000002},"last":0.755},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":4.0640000000000001,"5min":4.0640000000000001,"15min":4.0640000000000001},"last":1.0069999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":3.4289999999999998,"5min":3.4289999999999998,"15min":3.4289999999999998},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.464,"15min":1.464},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.0289999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47092,"kb_used_data":28972,"kb_used_omap":0,"kb_used_meta":18112,"kb_avail":94324748,"statfs":{"total":96636764160,"available":96588541952,"internally_reserved":0,"allocated":29667328,"data_stored":43957433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18546688},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.1269999999999998,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":0.749}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8799999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":10.584,"5min":10.584,"15min":10.584},"last":0.93700000000000006}]},{"osd":4,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.5329999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.359},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.387}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.206},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.4350000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"min":{"1min":1.3,"5min":1.3,"15min":1.3},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.6220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60168,"kb_used_data":43008,"kb_used_omap":0,"kb_used_meta":17152,"kb_avail":94311672,"statfs":{"total":96636764160,"available":96575152128,"internally_reserved":0,"allocated":44040192,"data_stored":54400664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17563648},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.04}]},{"osd":3,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1819999999999999}]},{"osd":4,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.244},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.452},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.4099999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.407}]},{"osd":7,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.546}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56892,"kb_used_data":39156,"kb_used_omap":0,"kb_used_meta":17728,"kb_avail":94314948,"statfs":{"total":96636764160,"available":96578506752,"internally_reserved":0,"allocated":40095744,"data_stored":53423289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18153472},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.94599999999999995}]},{"osd":1,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.038}]},{"osd":3,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.752},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.97099999999999997}]},{"osd":4,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.421},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.091},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.613}]},{"osd":7,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"min":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":21664,"kb_used_data":6616,"kb_used_omap":0,"kb_used_meta":15040,"kb_avail":94350176,"statfs":{"total":96636764160,"available":96614580224,"internally_reserved":0,"allocated":6774784,"data_stored":9773247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15400960},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.98499999999999999}]},{"osd":1,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.069},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.2190000000000001}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":0.94899999999999995}]},{"osd":4,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.179},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.4870000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.4330000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.641}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66108,"kb_used_data":44276,"kb_used_omap":0,"kb_used_meta":21824,"kb_avail":94305732,"statfs":{"total":96636764160,"available":96569069568,"internally_reserved":0,"allocated":45338624,"data_stored":65682617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":22347776},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.3420000000000001}]},{"osd":2,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.161,"5min":2.161,"15min":2.161},"last":1.409}]},{"osd":3,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.109}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0249999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.577,"15min":1.577},"min":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":1.27},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.137}]}]},{"osd":5,"up_from":0,"seq":55834574872,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}]}} 2021-01-15T19:09:40.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pgp_num 11 2021-01-15T19:09:40.748 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:40.748 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:40.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:40.791 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:40.790+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:40.892 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:40.891+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:40.933 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:40.933 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:40.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:40.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:41.027 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:41.026+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:41.093 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:41.092+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:41.119 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:41.120 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:41.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:41.132 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:41.154 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:41.175 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:41.195 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:41.193+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:41.195 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:41.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:41.238 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:41.247 INFO:tasks.rados.rados.0.smithi001.stdout:440: done (2 left) 2021-01-15T19:09:41.247 INFO:tasks.rados.rados.0.smithi001.stdout:442: done (1 left) 2021-01-15T19:09:41.247 INFO:tasks.rados.rados.0.smithi001.stdout:443: done (0 left) 2021-01-15T19:09:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:444: rmattr oid 10 current snap is 46 2021-01-15T19:09:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:444: done (0 left) 2021-01-15T19:09:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:445: rmattr oid 33 current snap is 46 2021-01-15T19:09:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:446: snap_create 2021-01-15T19:09:41.251 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 1 pgp_num to 11 2021-01-15T19:09:41.257 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 156 (ObjNum 138 snap 34 seq_num 138) dirty exists 2021-01-15T19:09:41.268 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:41.270 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:41.271 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:41.271 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:41.278 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:41.278 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:41.279 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:41.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:41.331 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:41.329+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:41.396 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:41.395+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:41.464 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:41.464 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:41.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:41.464 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:41.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.465 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:41.465 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:41.465 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.212588255999997, 2021-01-15T19:09:41.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:09:41.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.466 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.467 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:09:41.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.467 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:41.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:09:41.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:09:41.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:09:41.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:09:41.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:09:41.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:09:41.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:09:41.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:09:41.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:09:41.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:09:41.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:09:41.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:09:41.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:09:41.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:09:41.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:09:41.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:09:41.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:09:41.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:09:41.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:09:41.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:09:41.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:09:41.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:09:41.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.481 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.482 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:41.482 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:41.483 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.212397312, 2021-01-15T19:09:41.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:09:41.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:41.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.484 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:09:41.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.485 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:41.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:09:41.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:09:41.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:09:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:09:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:09:41.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:09:41.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:09:41.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:09:41.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:09:41.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:09:41.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:09:41.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:09:41.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:41.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:09:41.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:09:41.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:09:41.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:09:41.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:41.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:09:41.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:09:41.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:09:41.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:09:41.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:09:41.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:09:41.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:09:41.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:09:41.502 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.503 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.504 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.504 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:41.505 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.273445215000002, 2021-01-15T19:09:41.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:09:41.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:09:41.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.508 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:41.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:09:41.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:09:41.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:09:41.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:09:41.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:09:41.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:09:41.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:09:41.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:09:41.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:09:41.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:09:41.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:09:41.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:09:41.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:09:41.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:09:41.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:09:41.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:09:41.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:09:41.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:09:41.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:09:41.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:09:41.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:09:41.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:09:41.525 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.525 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.526 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:41.527 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:41.527 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.270724754, 2021-01-15T19:09:41.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:09:41.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:41.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:09:41.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.529 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:41.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:09:41.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:09:41.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:09:41.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:09:41.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:09:41.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:09:41.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:09:41.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:09:41.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:09:41.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:09:41.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:09:41.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:09:41.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:41.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:09:41.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:09:41.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:09:41.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:09:41.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:41.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:09:41.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:09:41.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:09:41.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:09:41.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:09:41.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:09:41.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:09:41.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:09:41.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:41.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.270612174999997, 2021-01-15T19:09:41.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:09:41.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.548 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:09:41.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.548 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:09:41.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:09:41.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:09:41.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:09:41.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:09:41.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:09:41.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:09:41.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:09:41.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:09:41.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:09:41.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:09:41.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:09:41.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:09:41.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:09:41.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:09:41.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:09:41.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:09:41.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:09:41.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:09:41.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:09:41.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:09:41.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:09:41.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.567 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:41.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:41.570 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.270482270999999, 2021-01-15T19:09:41.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:09:41.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:41.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:09:41.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:41.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:09:41.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:09:41.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:09:41.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:09:41.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:09:41.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:09:41.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:09:41.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:09:41.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:09:41.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:09:41.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:09:41.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:09:41.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:41.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:09:41.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:09:41.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:09:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:09:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:09:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:09:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:09:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:09:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:09:41.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:09:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:09:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:09:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.592 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.594 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.247277001999997, 2021-01-15T19:09:41.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:09:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.597 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:09:41.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.599 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:41.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:09:41.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:09:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:09:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:09:41.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:09:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:09:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:09:41.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:09:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:09:41.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:09:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:09:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:09:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:09:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:09:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:09:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:09:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:09:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:09:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:09:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:09:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:09:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:09:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.620 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.247158804000001, 2021-01-15T19:09:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:09:41.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:09:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:09:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:09:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:09:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:09:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:09:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:09:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:09:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:09:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:09:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:09:41.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:09:41.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:09:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:09:41.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:09:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:09:41.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:09:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:09:41.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:09:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:09:41.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:09:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:09:41.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:09:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.247046114, 2021-01-15T19:09:41.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:09:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.639 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:09:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:09:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:09:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:09:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:09:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:09:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:09:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:09:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:09:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:09:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:09:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:09:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:09:41.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:09:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:09:41.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:09:41.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:09:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:09:41.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:09:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:09:41.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:09:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:09:41.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:09:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:09:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.241444364000003, 2021-01-15T19:09:41.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:09:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:09:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:09:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:09:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:09:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:09:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:09:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:09:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:09:41.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:09:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:09:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:09:41.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:09:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:09:41.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:09:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.232975639999999, 2021-01-15T19:09:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:09:41.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:09:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:09:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:09:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:09:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:09:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:09:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:09:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:09:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:09:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:09:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:09:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:09:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:09:41.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:09:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:09:41.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:09:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:09:41.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:09:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:09:41.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:09:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:09:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:09:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:09:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:41.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.228457057, 2021-01-15T19:09:41.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:09:41.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.684 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:09:41.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.684 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:41.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:09:41.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:09:41.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:09:41.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:09:41.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:09:41.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:09:41.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:09:41.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:09:41.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:09:41.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:09:41.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:09:41.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:09:41.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:09:41.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:09:41.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:09:41.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:09:41.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:09:41.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:09:41.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:09:41.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:09:41.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:09:41.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:09:41.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.698 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.699 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:569 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~581632] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.699 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:41.699 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.224049960000002, 2021-01-15T19:09:41.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002, 2021-01-15T19:09:41.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.701 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 569 2021-01-15T19:09:41.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.701 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:41.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237431+0000", 2021-01-15T19:09:41.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039774999999999999 2021-01-15T19:09:41.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237829+0000", 2021-01-15T19:09:41.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6169999999999996e-06 2021-01-15T19:09:41.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237839+0000", 2021-01-15T19:09:41.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6092000000000001e-05 2021-01-15T19:09:41.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237855+0000", 2021-01-15T19:09:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2770000000000005e-06 2021-01-15T19:09:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237863+0000", 2021-01-15T19:09:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097738170000000006 2021-01-15T19:09:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247637+0000", 2021-01-15T19:09:41.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00058925299999999998 2021-01-15T19:09:41.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248226+0000", 2021-01-15T19:09:41.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063443099999999997 2021-01-15T19:09:41.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.248861+0000", 2021-01-15T19:09:41.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0067300490000000001 2021-01-15T19:09:41.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255591+0000", 2021-01-15T19:09:41.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037689989999999999 2021-01-15T19:09:41.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259360+0000", 2021-01-15T19:09:41.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2400999999999999e-05 2021-01-15T19:09:41.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259442+0000", 2021-01-15T19:09:41.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022010686000000002 2021-01-15T19:09:41.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.714 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.715 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:570 3.9s0 3:940930a3:::smithi00140323-8:head [setxattr _header (56) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:09:41.716 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:41.716 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.223567834000001, 2021-01-15T19:09:41.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998, 2021-01-15T19:09:41.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 570 2021-01-15T19:09:41.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.718 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:41.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237913+0000", 2021-01-15T19:09:41.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2658000000000003e-05 2021-01-15T19:09:41.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237966+0000", 2021-01-15T19:09:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2420000000000003e-06 2021-01-15T19:09:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237973+0000", 2021-01-15T19:09:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3407999999999999e-05 2021-01-15T19:09:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237987+0000", 2021-01-15T19:09:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.263e-06 2021-01-15T19:09:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.237993+0000", 2021-01-15T19:09:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013261488 2021-01-15T19:09:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251255+0000", 2021-01-15T19:09:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012682000000000001 2021-01-15T19:09:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251381+0000", 2021-01-15T19:09:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051990599999999999 2021-01-15T19:09:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.251901+0000", 2021-01-15T19:09:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0036923379999999999 2021-01-15T19:09:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.255594+0000", 2021-01-15T19:09:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0041920639999999997 2021-01-15T19:09:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259786+0000", 2021-01-15T19:09:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8251000000000005e-05 2021-01-15T19:09:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259884+0000", 2021-01-15T19:09:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021970437999999998 2021-01-15T19:09:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:09:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.223439288000002, 2021-01-15T19:09:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:09:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:09:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:09:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:09:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:09:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:09:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:09:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:09:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:09:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:09:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:09:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:09:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:09:41.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:09:41.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:41.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:09:41.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:09:41.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:09:41.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:09:41.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:09:41.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:09:41.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:09:41.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:09:41.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.747 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:09:41.747 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:41.747 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.214990147999998, 2021-01-15T19:09:41.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:09:41.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:41.748 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:41.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:09:41.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:09:41.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:09:41.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:09:41.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:09:41.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:09:41.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:09:41.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:09:41.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:09:41.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:09:41.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:09:41.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:09:41.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:09:41.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:09:41.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.757 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.758 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:09:41.758 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:41.758 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.256315209999997, 2021-01-15T19:09:41.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:09:41.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:09:41.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.760 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:41.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:09:41.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:09:41.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:09:41.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:09:41.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:09:41.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:09:41.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:09:41.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:09:41.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:09:41.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:09:41.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:09:41.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:09:41.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:41.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:09:41.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:09:41.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:09:41.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:09:41.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:09:41.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:09:41.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:09:41.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:09:41.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:09:41.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:09:41.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:09:41.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:09:41.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:09:41.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:09:41.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.776 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.12s1 74/57 ECSubWrite(tid=175, reqid=client.4418.0:742, at_version=74'42, trim_to=0'0, roll_forward_to=74'40))", 2021-01-15T19:09:41.777 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:41.777 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.132130527999998, 2021-01-15T19:09:41.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686, 2021-01-15T19:09:41.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:41.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:41.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.329351+0000", 2021-01-15T19:09:41.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043390989999999999 2021-01-15T19:09:41.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333690+0000", 2021-01-15T19:09:41.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4856000000000001e-05 2021-01-15T19:09:41.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333715+0000", 2021-01-15T19:09:41.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8349e-05 2021-01-15T19:09:41.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333733+0000", 2021-01-15T19:09:41.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9689999999999994e-06 2021-01-15T19:09:41.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333741+0000", 2021-01-15T19:09:41.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3493999999999999e-05 2021-01-15T19:09:41.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333805+0000", 2021-01-15T19:09:41.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5466999999999999e-05 2021-01-15T19:09:41.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.333820+0000", 2021-01-15T19:09:41.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015347355 2021-01-15T19:09:41.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349167+0000", 2021-01-15T19:09:41.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0270999999999999e-05 2021-01-15T19:09:41.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:02.349228+0000", 2021-01-15T19:09:41.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01987686 2021-01-15T19:09:41.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:762 3.ds0 3:bd2d5ff7:::smithi00140323-2:2a [read 0~2613248,checksum] snapc 0=[] ondisk+read+known_if_redirected e76)", 2021-01-15T19:09:41.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:41.791 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.27076924, 2021-01-15T19:09:41.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002, 2021-01-15T19:09:41.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:41.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 762 2021-01-15T19:09:41.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.793 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:41.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190712+0000", 2021-01-15T19:09:41.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4570999999999997e-05 2021-01-15T19:09:41.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190757+0000", 2021-01-15T19:09:41.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2950000000000004e-06 2021-01-15T19:09:41.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190761+0000", 2021-01-15T19:09:41.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2810000000000003e-06 2021-01-15T19:09:41.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190768+0000", 2021-01-15T19:09:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5410000000000001e-06 2021-01-15T19:09:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.190772+0000", 2021-01-15T19:09:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00093814699999999996 2021-01-15T19:09:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191710+0000", 2021-01-15T19:09:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111131 2021-01-15T19:09:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.191821+0000", 2021-01-15T19:09:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020833550999999999 2021-01-15T19:09:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:04.212655+0000", 2021-01-15T19:09:41.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021942517000000002 2021-01-15T19:09:41.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.803 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:09:41.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:41.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.279128561999997, 2021-01-15T19:09:41.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:09:41.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:41.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:41.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:41.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:41.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:41.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:09:41.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:41.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:41.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:41.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:41.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:41.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:09:41.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:09:41.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:41.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:09:41.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:09:41.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:41.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:09:41.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:09:41.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:41.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:09:41.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:09:41.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:41.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:09:41.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:09:41.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:09:41.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:09:41.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:41.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:09:41.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:09:41.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:41.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:09:41.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:09:41.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:41.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:09:41.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:09:41.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:41.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:09:41.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:09:41.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:41.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:09:41.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:09:41.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:41.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:09:41.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:09:41.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:41.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:41.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:41.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:09:41.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:09:41.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.821 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:41.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:41.822 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:41.825 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:41.823+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:41.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:41.926 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:41.925+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:41.963 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:41.963 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:41.973 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:42.029 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:42.027+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:42.130 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:42.129+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:42.140 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:42.140 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:42.141 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:42.141 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:42.142 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:42.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:42.231 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:42.230+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:42.252 INFO:tasks.rados.rados.0.smithi001.stdout:445: done (1 left) 2021-01-15T19:09:42.253 INFO:tasks.rados.rados.0.smithi001.stdout:446: done (0 left) 2021-01-15T19:09:42.253 INFO:tasks.rados.rados.0.smithi001.stdout:447: rollback oid 36 current snap is 47 2021-01-15T19:09:42.253 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 36 to 36 2021-01-15T19:09:42.254 INFO:tasks.rados.rados.0.smithi001.stdout:448: copy_from oid 27 from oid 15 current snap is 47 2021-01-15T19:09:42.254 INFO:tasks.rados.rados.0.smithi001.stdout:449: rmattr oid 31 current snap is 47 2021-01-15T19:09:42.254 INFO:tasks.rados.rados.0.smithi001.stdout:449: done (2 left) 2021-01-15T19:09:42.254 INFO:tasks.rados.rados.0.smithi001.stdout:450: setattr oid 1 current snap is 47 2021-01-15T19:09:42.255 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 28 current snap is 47 2021-01-15T19:09:42.255 INFO:tasks.rados.rados.0.smithi001.stdout:451: seq_num 167 ranges {2949120=401408} 2021-01-15T19:09:42.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:09:42.275 INFO:tasks.rados.rados.0.smithi001.stdout:451: writing smithi00140323-28 from 2949120 to 3350528 tid 1 2021-01-15T19:09:42.276 INFO:tasks.rados.rados.0.smithi001.stdout:447: finishing rollback tid 0 to smithi00140323-36 2021-01-15T19:09:42.276 INFO:tasks.rados.rados.0.smithi001.stdout:447: finishing rollback tid 1 to smithi00140323-36 2021-01-15T19:09:42.277 INFO:tasks.rados.rados.0.smithi001.stdout:447: finishing rollback tid 2 to smithi00140323-36 2021-01-15T19:09:42.277 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 119 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:42.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 120 (ObjNum 742092112 snap 32581 seq_num 179666464) dirty exists 2021-01-15T19:09:42.288 INFO:tasks.rados.rados.0.smithi001.stdout:447: done (3 left) 2021-01-15T19:09:42.288 INFO:tasks.rados.rados.0.smithi001.stdout:450: done (2 left) 2021-01-15T19:09:42.289 INFO:tasks.rados.rados.0.smithi001.stdout:452: snap_create 2021-01-15T19:09:42.290 INFO:tasks.rados.rados.0.smithi001.stdout:451: finishing write tid 1 to smithi00140323-28 2021-01-15T19:09:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:451: finishing write tid 2 to smithi00140323-28 2021-01-15T19:09:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:451: finishing write tid 3 to smithi00140323-28 2021-01-15T19:09:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 205 (ObjNum 167 snap 47 seq_num 167) dirty exists 2021-01-15T19:09:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:451: left oid 28 (ObjNum 167 snap 47 seq_num 167) 2021-01-15T19:09:42.295 INFO:tasks.rados.rados.0.smithi001.stdout:448: finishing copy_from to smithi00140323-27 2021-01-15T19:09:42.296 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 27 v 54 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:09:42.296 INFO:tasks.rados.rados.0.smithi001.stdout:448: finishing copy_from racing read to smithi00140323-27 2021-01-15T19:09:42.299 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.032173683000003, 2021-01-15T19:09:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:42.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:42.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:42.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:42.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:42.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:42.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:42.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.115501180999999, 2021-01-15T19:09:42.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:42.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:42.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:42.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:42.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:42.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:42.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:42.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.336 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.115437381, 2021-01-15T19:09:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:42.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:42.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:42.350 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.351 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.112854615000003, 2021-01-15T19:09:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.352 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:42.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:42.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:42.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:42.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:42.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:42.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:42.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:42.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:42.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.365 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.366 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:42.366 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:42.366 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.112691673999997, 2021-01-15T19:09:42.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:42.367 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.367 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:42.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:42.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:42.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:42.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:42.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:42.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:42.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:42.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:42.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:42.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:42.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:42.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:42.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:42.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:42.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:42.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:42.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:42.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.380 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.380 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.380 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:42.381 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:42.381 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.951649027999999, 2021-01-15T19:09:42.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:42.381 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.381 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.382 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.382 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:42.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.383 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:42.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:42.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:42.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:42.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:42.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:42.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:42.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:42.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:42.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:42.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:42.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:42.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:42.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:42.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:42.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:42.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:42.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:42.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:42.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:42.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:42.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:42.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:42.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:42.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:42.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:42.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:42.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.400 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:42.400 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:42.400 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.951541572, 2021-01-15T19:09:42.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:42.401 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.401 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.402 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:42.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.402 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:42.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:42.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:42.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:42.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:42.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:42.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:42.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:42.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:42.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:42.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:42.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:42.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:42.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:42.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:42.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:42.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:42.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:42.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:42.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:42.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:42.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:42.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:42.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:42.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:42.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:42.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:42.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:42.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:42.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:42.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:42.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.818474706, 2021-01-15T19:09:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.429 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:42.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.429 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:42.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:42.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:42.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:42.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:42.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:42.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:42.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:42.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:42.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:42.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:42.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:42.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:42.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:42.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:42.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:42.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.457 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.458 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:42.459 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:42.459 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.922507023000001, 2021-01-15T19:09:42.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:42.460 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.461 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:42.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.461 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.462 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:42.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:42.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:42.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:42.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:42.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:42.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:42.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:42.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:42.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:42.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:42.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:42.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:42.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:42.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:42.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:42.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:42.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:42.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:42.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:42.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.485 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.487 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:42.487 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:42.487 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.922376892999999, 2021-01-15T19:09:42.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:42.488 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.488 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:42.488 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.488 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.489 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:42.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.489 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:42.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:42.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:42.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:42.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:42.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:42.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:42.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:42.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:42.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:42.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:42.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:42.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:42.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:42.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:42.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:42.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:42.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:42.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:42.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.503 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.505 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:42.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:42.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.922291483999999, 2021-01-15T19:09:42.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:42.506 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:42.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.508 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.508 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:42.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.509 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:42.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:42.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:42.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:42.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:42.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:42.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:42.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:42.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:42.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:42.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:42.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:42.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:42.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:42.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:42.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:42.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:42.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:42.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:42.524 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.527 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:42.527 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:42.527 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.091117096000001, 2021-01-15T19:09:42.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:09:42.528 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.528 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.528 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.528 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.529 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.529 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:42.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.529 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:42.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:42.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:42.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:42.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:42.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:42.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:42.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:42.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:42.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:42.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:42.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:42.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:42.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:42.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:42.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:42.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:42.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:42.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:42.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:09:42.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:09:42.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:09:42.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:09:42.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:09:42.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:09:42.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:09:42.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:09:42.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:09:42.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:09:42.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:09:42.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:09:42.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:09:42.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.553 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:42.554 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:42.554 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.091028446999999, 2021-01-15T19:09:42.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:09:42.555 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.555 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:42.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:42.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:42.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:42.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:42.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:42.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:42.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:09:42.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:09:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:09:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:09:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:09:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:09:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:09:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:09:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:09:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:09:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:09:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:09:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:09:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:994 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.081954366000001, 2021-01-15T19:09:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998, 2021-01-15T19:09:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.577 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.577 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.577 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 994 2021-01-15T19:09:42.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.577 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3285999999999999e-05 2021-01-15T19:09:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214805+0000", 2021-01-15T19:09:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2021-01-15T19:09:42.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214818+0000", 2021-01-15T19:09:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5945000000000001e-05 2021-01-15T19:09:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214833+0000", 2021-01-15T19:09:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214840+0000", 2021-01-15T19:09:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9747999999999998e-05 2021-01-15T19:09:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214900+0000", 2021-01-15T19:09:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146553 2021-01-15T19:09:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215046+0000", 2021-01-15T19:09:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081407799999999998 2021-01-15T19:09:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215860+0000", 2021-01-15T19:09:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095697720000000007 2021-01-15T19:09:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225430+0000", 2021-01-15T19:09:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012440451 2021-01-15T19:09:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237871+0000", 2021-01-15T19:09:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9866999999999998e-05 2021-01-15T19:09:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237970+0000", 2021-01-15T19:09:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998 2021-01-15T19:09:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.592 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:995 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 3e] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:42.592 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:42.592 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.081834589, 2021-01-15T19:09:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001, 2021-01-15T19:09:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 995 2021-01-15T19:09:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8172e-05 2021-01-15T19:09:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214919+0000", 2021-01-15T19:09:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4777999999999999e-05 2021-01-15T19:09:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214944+0000", 2021-01-15T19:09:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4625e-05 2021-01-15T19:09:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214959+0000", 2021-01-15T19:09:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2830000000000001e-06 2021-01-15T19:09:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214967+0000", 2021-01-15T19:09:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002508929 2021-01-15T19:09:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217476+0000", 2021-01-15T19:09:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8955000000000006e-05 2021-01-15T19:09:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217545+0000", 2021-01-15T19:09:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066629600000000003 2021-01-15T19:09:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218211+0000", 2021-01-15T19:09:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072214000000000002 2021-01-15T19:09:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225432+0000", 2021-01-15T19:09:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012976164 2021-01-15T19:09:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238409+0000", 2021-01-15T19:09:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859000000000001e-05 2021-01-15T19:09:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238463+0000", 2021-01-15T19:09:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001 2021-01-15T19:09:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:996 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:42.609 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.081706103999998, 2021-01-15T19:09:42.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001, 2021-01-15T19:09:42.609 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.609 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 996 2021-01-15T19:09:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.611 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:42.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9251000000000001e-05 2021-01-15T19:09:42.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215049+0000", 2021-01-15T19:09:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:09:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215056+0000", 2021-01-15T19:09:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6064000000000001e-05 2021-01-15T19:09:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215072+0000", 2021-01-15T19:09:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:09:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215078+0000", 2021-01-15T19:09:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294249999999997 2021-01-15T19:09:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220107+0000", 2021-01-15T19:09:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4726000000000002e-05 2021-01-15T19:09:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220182+0000", 2021-01-15T19:09:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043078000000000002 2021-01-15T19:09:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220613+0000", 2021-01-15T19:09:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048209209999999997 2021-01-15T19:09:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225434+0000", 2021-01-15T19:09:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013153299 2021-01-15T19:09:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238587+0000", 2021-01-15T19:09:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5006000000000002e-05 2021-01-15T19:09:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238642+0000", 2021-01-15T19:09:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001 2021-01-15T19:09:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:09:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1075203889999994, 2021-01-15T19:09:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:09:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:09:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:09:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:09:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:09:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:09:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:09:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:09:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:09:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:09:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:09:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:09:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:09:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:09:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:09:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:09:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:09:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:09:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:09:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:09:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:09:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:09:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:09:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:09:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:09:42.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:09:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:09:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:09:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1073883139999996, 2021-01-15T19:09:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:09:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:09:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:09:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:09:42.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:42.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:09:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:09:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:09:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:09:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:09:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:09:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:09:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:09:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:09:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:09:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:09:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:09:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:09:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:09:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:09:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:09:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:09:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:09:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:09:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:09:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:09:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:09:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:09:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:09:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:09:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.104252797, 2021-01-15T19:09:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:09:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:09:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:09:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:09:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:09:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:09:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:09:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:09:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:09:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:09:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:09:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:09:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:09:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:09:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:09:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:09:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:09:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:09:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:09:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:09:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:09:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:09:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:09:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:09:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:09:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:09:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:09:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:09:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1041487710000002, 2021-01-15T19:09:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:09:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:09:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:42.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:42.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:09:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:09:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:09:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:09:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:09:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:09:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:09:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:09:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:09:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:09:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:09:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:09:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:09:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:09:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:09:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:09:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:09:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:09:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:09:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:09:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:09:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:09:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:09:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:09:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:09:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:09:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:09:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:42.702 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:42.704 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:42.704 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:42.702+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:42.804 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:42.803+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:42.882 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:42.883 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:42.893 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:42.905 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:42.904+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:43.006 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:43.005+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:43.017 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:43.018 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:43.018 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:43.018 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:43.018 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:43.027 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:43.107 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:43.106+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:43.197 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:43.197 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:43.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:43.198 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:43.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.198 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.908648513000003, 2021-01-15T19:09:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.199 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.200 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:43.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:43.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:43.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:43.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:43.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:43.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:43.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:43.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:43.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:43.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:43.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:43.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:43.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:43.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:43.215 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.215 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.216 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:43.217 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:43.217 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.976885746999997, 2021-01-15T19:09:43.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:43.217 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.217 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.218 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:43.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.219 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:43.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:43.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:43.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:43.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:43.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:43.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:43.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:43.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:43.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:43.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:43.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:43.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:43.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:43.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:43.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:43.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:43.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:43.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:43.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:43.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:43.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:43.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:43.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:43.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:43.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:43.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:43.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:43.233 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.233 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.233 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.234 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:43.234 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:43.239 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.974597705999997, 2021-01-15T19:09:43.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:43.239 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.240 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:43.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:43.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:43.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:43.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:43.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:43.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:43.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:43.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:43.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:43.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:43.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:43.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:43.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:43.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:43.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:43.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:43.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.258 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:43.258 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:43.258 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.972008080999998, 2021-01-15T19:09:43.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:43.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.260 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.260 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:43.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.260 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:43.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:43.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:43.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:43.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:43.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:43.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:43.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:43.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:43.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:43.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:43.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:43.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:43.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:43.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:43.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:43.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.274 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.971099803000001, 2021-01-15T19:09:43.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:43.276 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.278 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:43.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:43.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:43.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:43.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:43.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:43.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:43.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:43.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:43.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:43.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:43.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:43.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:43.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:43.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:43.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:43.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:43.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:43.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:43.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:43.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:43.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:43.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:43.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.292 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.293 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:43.293 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:43.293 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.002415896000002, 2021-01-15T19:09:43.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:43.294 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.294 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.294 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.295 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:43.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.295 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:43.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:43.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:43.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:43.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:43.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:43.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:43.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:43.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:43.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:43.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:43.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:43.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:43.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:43.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:43.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:43.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:43.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:43.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:43.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:43.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:43.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:43.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:43.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.002266159000001, 2021-01-15T19:09:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:43.312 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.313 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:43.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:43.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:43.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:43.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:43.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:43.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:43.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:43.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:43.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:43.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:43.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:43.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:43.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:43.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:43.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:43.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:43.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:43.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:43.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:43.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:43.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:43.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:43.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:43.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:43.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:43.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:43.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:43.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:43.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:43.331 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.331 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.331 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.332 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:43.332 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:43.332 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.002114261000003, 2021-01-15T19:09:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.333 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.334 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:43.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.334 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:43.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:43.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:43.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:43.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:43.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:43.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:43.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:43.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:43.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:43.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:43.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:43.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:43.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:43.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:43.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:43.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:43.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:43.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:43.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:43.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:43.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:43.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:43.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.001988948999994, 2021-01-15T19:09:43.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:43.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:43.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.368 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:43.368 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:43.369 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.981156373999994, 2021-01-15T19:09:43.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:43.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.370 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:43.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:43.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:43.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:43.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:43.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:43.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:43.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:43.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:43.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:43.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:43.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.974863041999996, 2021-01-15T19:09:43.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:43.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:43.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:43.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:43.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:43.398 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.012803528999996, 2021-01-15T19:09:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:43.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:43.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:43.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.950431549000001, 2021-01-15T19:09:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.416 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:43.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:43.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:43.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:43.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:43.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:43.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:43.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:43.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:43.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:43.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:43.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:43.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.947129048000001, 2021-01-15T19:09:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:43.431 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.432 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.432 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:43.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.433 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:43.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:43.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:43.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:43.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:43.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:43.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.945807182000003, 2021-01-15T19:09:43.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:43.447 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.447 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.447 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:43.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:43.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:43.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:43.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:43.461 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.945436929000003, 2021-01-15T19:09:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.462 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.463 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:43.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:43.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:43.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:43.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:43.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:43.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:43.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:43.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:43.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:43.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:43.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.476 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.945102536, 2021-01-15T19:09:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.477 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.478 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:43.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:43.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:43.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:43.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:43.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:43.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:43.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:43.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:43.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:43.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:43.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:43.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:43.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:43.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:43.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:43.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:43.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 50.003841276999999, 2021-01-15T19:09:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.011478404999998, 2021-01-15T19:09:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:43.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:43.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:43.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:43.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:43.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:43.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:43.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:43.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:43.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:43.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:43.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:43.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:43.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:43.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:43.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.521 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:43.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.981484201000001, 2021-01-15T19:09:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:43.523 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:43.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:43.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:43.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:43.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:43.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:43.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:43.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:43.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:43.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:43.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:43.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:43.537 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:43.537 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:43.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:43.539 INFO:tasks.rados.rados.0.smithi001.stdout:448: done (2 left) 2021-01-15T19:09:43.540 INFO:tasks.rados.rados.0.smithi001.stdout:451: done (1 left) 2021-01-15T19:09:43.540 INFO:tasks.rados.rados.0.smithi001.stdout:452: done (0 left) 2021-01-15T19:09:43.540 INFO:tasks.rados.rados.0.smithi001.stdout:453: snap_create 2021-01-15T19:09:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:09:43.541 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:43.537+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:43.541 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:43.642 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:43.638+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:43.679 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:43.680 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:43.680 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:43.680 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:43.689 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:43.739 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:43.740 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:43.738+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:43.759 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:43.779 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:43.799 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:43.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:43.840 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:43.839+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:43.841 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:43.855 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:43.856 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:43.856 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:43.856 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:43.856 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:43.862 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:43.862 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.863 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.863 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.863 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.863 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.864 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.864 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:43.864 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:43.864 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:43.865 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:43.865 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:43.867 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:43.941 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:43.940+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:43.994 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:43.994 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:44.042 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:44.041+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:44.137 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:44.137 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.143 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:44.142+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:44.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:44.244 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:44.243+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:44.260 INFO:tasks.rados.rados.0.smithi001.stdout:453: done (0 left) 2021-01-15T19:09:44.261 INFO:tasks.rados.rados.0.smithi001.stdout:454: read oid 1 snap -1 2021-01-15T19:09:44.261 INFO:tasks.rados.rados.0.smithi001.stdout:454: expect (ObjNum 0 snap 0 seq_num 0) 2021-01-15T19:09:44.261 INFO:tasks.rados.rados.0.smithi001.stdout:455: setattr oid 49 current snap is 49 2021-01-15T19:09:44.261 INFO:tasks.rados.rados.0.smithi001.stdout:456: snap_remove snap 42 2021-01-15T19:09:44.284 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:44.284 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:44.345 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:44.344+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:44.427 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:44.427 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:44.479 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:44.477+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:44.556 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:44.551+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:44.652 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:44.652 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.652 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:44.656 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:44.655+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:44.755 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:44.753+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:44.792 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:44.793 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.803 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:44.860 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:44.858+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:44.935 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:44.936 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:44.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:44.957 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:44.955+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:45.057 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:45.055+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:45.085 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:45.085 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:45.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:45.188 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:45.187+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:45.261 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:45.259+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:45.263 INFO:tasks.rados.rados.0.smithi001.stdout:456: done (2 left) 2021-01-15T19:09:45.263 INFO:tasks.rados.rados.0.smithi001.stdout:457: delete oid 3 current snap is 49 2021-01-15T19:09:45.264 INFO:tasks.rados.rados.0.smithi001.stdout:454: expect (ObjNum 0 snap 0 seq_num 0) 2021-01-15T19:09:45.265 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 94 (ObjNum 122 snap 27 seq_num 122) dirty exists 2021-01-15T19:09:45.270 INFO:tasks.rados.rados.0.smithi001.stdout:454: done (2 left) 2021-01-15T19:09:45.271 INFO:tasks.rados.rados.0.smithi001.stdout:455: done (1 left) 2021-01-15T19:09:45.271 INFO:tasks.rados.rados.0.smithi001.stdout:457: done (0 left) 2021-01-15T19:09:45.271 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 33 current snap is 49 2021-01-15T19:09:45.272 INFO:tasks.rados.rados.0.smithi001.stdout:458: seq_num 169 ranges {4366336=434176} 2021-01-15T19:09:45.282 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:45.283 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:45.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:45.306 INFO:tasks.rados.rados.0.smithi001.stdout:458: writing smithi00140323-33 from 4366336 to 4800512 tid 1 2021-01-15T19:09:45.307 INFO:tasks.rados.rados.0.smithi001.stdout:459: setattr oid 8 current snap is 49 2021-01-15T19:09:45.310 INFO:tasks.rados.rados.0.smithi001.stdout:460: snap_remove snap 48 2021-01-15T19:09:45.359 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:45.358+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:45.462 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:45.460+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:45.477 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:45.478 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:45.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:45.543 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:09:45.587 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:45.586+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:45.623 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:45.624 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:45.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:45.662 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:45.661+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:45.767 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:45.762+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:45.809 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:45.809 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:45.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:45.864 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:45.863+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:45.866 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:45.953 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:45.953 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:45.964 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:45.965 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:45.964+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:46.015 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:46.034 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:46.054 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:46.066 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:46.066+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:46.074 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:46.094 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:46.102 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:46.103 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:46.103 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:46.103 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:46.113 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:46.115 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:46.136 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.137 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:46.138 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:46.139 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:46.139 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:46.140 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:46.197 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:46.196+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:46.267 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2021-01-15T19:09:46.267 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:46.268 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 6, 7] 2021-01-15T19:09:46.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2021-01-15T19:09:46.270 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:46.268+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:46.273 INFO:tasks.rados.rados.0.smithi001.stdout:460: done (2 left) 2021-01-15T19:09:46.274 INFO:tasks.rados.rados.0.smithi001.stdout:461: rmattr oid 34 current snap is 49 2021-01-15T19:09:46.274 INFO:tasks.rados.rados.0.smithi001.stdout:461: done (2 left) 2021-01-15T19:09:46.274 INFO:tasks.rados.rados.0.smithi001.stdout:462: read oid 1 snap 43 2021-01-15T19:09:46.274 INFO:tasks.rados.rados.0.smithi001.stdout:462: expect deleted 2021-01-15T19:09:46.275 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 49 2021-01-15T19:09:46.275 INFO:tasks.rados.rados.0.smithi001.stdout:463: seq_num 170 ranges {352256=450560} 2021-01-15T19:09:46.281 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:09:46.281 INFO:tasks.rados.rados.0.smithi001.stdout:463: writing smithi00140323-39 from 352256 to 802816 tid 1 2021-01-15T19:09:46.282 INFO:tasks.rados.rados.0.smithi001.stdout:464: snap_create 2021-01-15T19:09:46.282 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 121 (ObjNum 131 snap 31 seq_num 131) dirty exists 2021-01-15T19:09:46.282 INFO:tasks.rados.rados.0.smithi001.stdout:458: finishing write tid 1 to smithi00140323-33 2021-01-15T19:09:46.282 INFO:tasks.rados.rados.0.smithi001.stdout:458: finishing write tid 2 to smithi00140323-33 2021-01-15T19:09:46.282 INFO:tasks.rados.rados.0.smithi001.stdout:458: finishing write tid 3 to smithi00140323-33 2021-01-15T19:09:46.283 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 159 (ObjNum 169 snap 49 seq_num 169) dirty exists 2021-01-15T19:09:46.283 INFO:tasks.rados.rados.0.smithi001.stdout:458: left oid 33 (ObjNum 169 snap 49 seq_num 169) 2021-01-15T19:09:46.283 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:46.284 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:46.284 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:46.284 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:46.284 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:46.286 INFO:tasks.rados.rados.0.smithi001.stdout:463: finishing write tid 1 to smithi00140323-39 2021-01-15T19:09:46.287 INFO:tasks.rados.rados.0.smithi001.stdout:463: finishing write tid 2 to smithi00140323-39 2021-01-15T19:09:46.288 INFO:tasks.rados.rados.0.smithi001.stdout:463: finishing write tid 3 to smithi00140323-39 2021-01-15T19:09:46.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 53 (ObjNum 170 snap 49 seq_num 170) dirty exists 2021-01-15T19:09:46.288 INFO:tasks.rados.rados.0.smithi001.stdout:463: left oid 39 (ObjNum 170 snap 49 seq_num 170) 2021-01-15T19:09:46.294 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:46.371 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:46.370+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:46.463 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:46.463 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:46.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:46.463 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:46.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.464 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:46.464 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:46.464 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.195675882000003, 2021-01-15T19:09:46.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:46.465 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.465 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.466 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:46.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.466 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:46.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:46.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:46.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:46.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:46.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:46.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:46.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:46.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:46.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:46.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:46.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:46.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:46.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:46.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:46.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:46.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:46.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:46.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:46.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:46.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:46.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:46.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:46.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:46.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:46.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:46.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:46.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.482 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.482 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.483 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:46.483 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:46.483 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.279003379999999, 2021-01-15T19:09:46.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:46.484 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.484 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.484 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.485 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:46.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.486 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:46.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:46.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:46.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:46.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:46.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:46.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:46.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:46.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:46.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:46.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:46.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:46.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:46.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:46.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:46.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:46.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:46.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:46.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:46.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:46.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:46.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:46.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.500 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.500 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.501 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:46.501 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:46.502 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.278939579999999, 2021-01-15T19:09:46.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:46.502 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.503 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.503 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.503 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:46.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.504 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:46.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:46.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:46.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:46.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:46.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:46.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:46.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:46.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:46.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:46.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:46.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:46.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:46.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:46.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:46.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:46.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:46.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:46.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:46.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:46.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:46.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:46.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.518 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:46.518 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:46.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.276356814000003, 2021-01-15T19:09:46.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:46.519 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.519 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.520 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:46.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.520 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:46.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:46.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:46.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:46.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:46.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:46.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:46.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:46.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:46.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:46.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:46.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:46.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:46.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:46.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:46.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:46.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:46.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:46.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:46.536 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.276193872999997, 2021-01-15T19:09:46.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:46.537 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.537 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.539 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:46.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.539 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:46.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:46.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:46.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:46.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:46.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:46.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:46.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:46.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:46.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:46.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:46.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:46.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:46.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:46.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:46.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:46.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:46.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.559 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:46.560 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:46.560 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.115151226999998, 2021-01-15T19:09:46.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:46.561 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.561 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.563 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:46.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.564 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:46.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:46.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:46.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:46.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:46.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:46.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:46.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:46.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:46.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:46.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:46.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:46.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:46.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:46.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:46.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:46.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:46.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:46.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:46.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:46.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:46.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:46.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:46.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:46.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:46.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:46.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:46.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:46.584 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:46.584 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.115043771000003, 2021-01-15T19:09:46.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:46.585 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.585 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.585 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.586 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:46.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.586 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:46.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:46.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:46.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:46.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:46.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:46.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:46.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:46.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:46.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:46.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:46.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:46.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:46.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:46.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:46.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:46.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:46.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:46.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:46.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:46.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:46.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:46.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:46.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:46.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:46.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:46.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:46.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:46.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:46.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:46.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:46.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.603 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:46.604 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:46.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.981976905000003, 2021-01-15T19:09:46.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:46.605 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.606 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.606 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:46.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:46.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:46.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:46.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:46.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:46.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:46.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:46.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:46.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:46.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:46.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:46.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:46.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:46.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:46.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:46.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:46.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:46.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:46.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:46.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:46.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:46.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:46.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:46.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:46.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:46.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:46.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:46.621 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.621 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.621 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.622 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:46.622 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:46.622 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.086009222000001, 2021-01-15T19:09:46.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:46.623 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.623 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:46.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.623 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.624 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.624 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:46.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:46.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:46.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:46.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:46.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:46.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:46.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:46.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:46.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:46.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:46.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:46.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:46.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:46.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:46.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:46.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:46.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:46.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:46.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:46.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.635 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:46.635 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:46.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.085879091999999, 2021-01-15T19:09:46.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:46.636 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.636 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:46.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.637 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:46.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.637 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:46.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:46.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:46.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:46.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:46.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:46.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:46.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:46.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:46.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:46.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:46.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:46.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:46.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:46.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:46.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:46.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:46.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:46.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:46.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.648 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:46.649 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:46.649 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.085793682999999, 2021-01-15T19:09:46.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:46.649 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.649 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:46.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.650 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:46.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:46.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:46.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:46.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:46.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:46.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:46.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:46.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:46.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:46.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:46.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:46.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:46.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:46.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:46.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:46.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:46.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:46.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:46.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:46.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.661 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.662 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:46.662 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:46.662 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.254619295000001, 2021-01-15T19:09:46.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:09:46.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.664 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:46.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:46.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:46.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:46.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:46.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:46.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:46.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:46.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:46.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:46.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:46.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:46.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:46.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:46.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:46.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:46.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:46.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:46.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:46.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:46.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:09:46.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:09:46.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:09:46.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:09:46.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:09:46.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:09:46.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:09:46.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:09:46.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:09:46.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:09:46.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:09:46.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:09:46.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:09:46.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.681 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:46.681 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:46.682 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.254530645999999, 2021-01-15T19:09:46.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:09:46.682 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.682 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.683 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.683 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:46.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.683 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:46.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:46.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:46.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:46.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:46.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:46.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:46.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:46.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:46.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:46.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:46.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:46.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:46.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:46.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:46.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:46.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:46.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:46.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:46.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:46.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:09:46.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:09:46.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:09:46.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:09:46.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:09:46.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:09:46.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:09:46.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:09:46.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:09:46.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:09:46.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:09:46.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:09:46.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:09:46.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.701 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:994 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:46.701 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:46.701 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.245456565000001, 2021-01-15T19:09:46.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998, 2021-01-15T19:09:46.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 994 2021-01-15T19:09:46.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.703 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:46.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:46.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3285999999999999e-05 2021-01-15T19:09:46.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214805+0000", 2021-01-15T19:09:46.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2021-01-15T19:09:46.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214818+0000", 2021-01-15T19:09:46.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5945000000000001e-05 2021-01-15T19:09:46.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214833+0000", 2021-01-15T19:09:46.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:46.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214840+0000", 2021-01-15T19:09:46.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9747999999999998e-05 2021-01-15T19:09:46.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214900+0000", 2021-01-15T19:09:46.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146553 2021-01-15T19:09:46.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215046+0000", 2021-01-15T19:09:46.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081407799999999998 2021-01-15T19:09:46.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215860+0000", 2021-01-15T19:09:46.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095697720000000007 2021-01-15T19:09:46.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225430+0000", 2021-01-15T19:09:46.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012440451 2021-01-15T19:09:46.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237871+0000", 2021-01-15T19:09:46.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9866999999999998e-05 2021-01-15T19:09:46.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237970+0000", 2021-01-15T19:09:46.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998 2021-01-15T19:09:46.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.716 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:995 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 3e] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:46.716 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:46.716 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.245336787999999, 2021-01-15T19:09:46.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001, 2021-01-15T19:09:46.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.718 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 995 2021-01-15T19:09:46.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.718 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:46.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:46.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8172e-05 2021-01-15T19:09:46.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214919+0000", 2021-01-15T19:09:46.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4777999999999999e-05 2021-01-15T19:09:46.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214944+0000", 2021-01-15T19:09:46.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4625e-05 2021-01-15T19:09:46.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214959+0000", 2021-01-15T19:09:46.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2830000000000001e-06 2021-01-15T19:09:46.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214967+0000", 2021-01-15T19:09:46.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002508929 2021-01-15T19:09:46.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217476+0000", 2021-01-15T19:09:46.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8955000000000006e-05 2021-01-15T19:09:46.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217545+0000", 2021-01-15T19:09:46.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066629600000000003 2021-01-15T19:09:46.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218211+0000", 2021-01-15T19:09:46.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072214000000000002 2021-01-15T19:09:46.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225432+0000", 2021-01-15T19:09:46.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012976164 2021-01-15T19:09:46.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238409+0000", 2021-01-15T19:09:46.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859000000000001e-05 2021-01-15T19:09:46.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238463+0000", 2021-01-15T19:09:46.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001 2021-01-15T19:09:46.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:996 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:46.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:46.732 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.245208302999998, 2021-01-15T19:09:46.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001, 2021-01-15T19:09:46.732 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.732 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.732 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.733 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.733 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.733 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 996 2021-01-15T19:09:46.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.733 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:46.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:46.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9251000000000001e-05 2021-01-15T19:09:46.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215049+0000", 2021-01-15T19:09:46.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:09:46.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215056+0000", 2021-01-15T19:09:46.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6064000000000001e-05 2021-01-15T19:09:46.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215072+0000", 2021-01-15T19:09:46.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:09:46.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215078+0000", 2021-01-15T19:09:46.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294249999999997 2021-01-15T19:09:46.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220107+0000", 2021-01-15T19:09:46.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4726000000000002e-05 2021-01-15T19:09:46.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220182+0000", 2021-01-15T19:09:46.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043078000000000002 2021-01-15T19:09:46.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220613+0000", 2021-01-15T19:09:46.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048209209999999997 2021-01-15T19:09:46.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225434+0000", 2021-01-15T19:09:46.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013153299 2021-01-15T19:09:46.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238587+0000", 2021-01-15T19:09:46.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5006000000000002e-05 2021-01-15T19:09:46.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238642+0000", 2021-01-15T19:09:46.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001 2021-01-15T19:09:46.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.746 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.747 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:09:46.747 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:46.747 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.271022587999999, 2021-01-15T19:09:46.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:09:46.747 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.748 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.748 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:09:46.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:46.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:46.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:09:46.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:09:46.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:09:46.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:09:46.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:09:46.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:09:46.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:09:46.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:09:46.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:09:46.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:09:46.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:09:46.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:46.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:09:46.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:09:46.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:09:46.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:09:46.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:09:46.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:09:46.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:09:46.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:09:46.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:09:46.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:09:46.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:09:46.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:09:46.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:09:46.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:09:46.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.763 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.764 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:09:46.764 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:46.764 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.270890512999999, 2021-01-15T19:09:46.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:09:46.765 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.765 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.765 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.765 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.765 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.766 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:09:46.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.766 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:46.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:46.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:09:46.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:09:46.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:46.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:09:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:09:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:09:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:09:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:09:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:09:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:09:46.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:09:46.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:09:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:09:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:09:46.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:09:46.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:46.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:09:46.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:09:46.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:09:46.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:09:46.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:46.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:09:46.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:09:46.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:09:46.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:09:46.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:09:46.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:09:46.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:09:46.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:09:46.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:09:46.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:46.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.2677549960000007, 2021-01-15T19:09:46.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:09:46.783 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.783 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:46.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.784 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:09:46.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.784 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:46.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:46.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:09:46.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:09:46.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:09:46.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:09:46.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:09:46.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:09:46.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:09:46.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:09:46.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:09:46.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:09:46.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:09:46.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:46.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:09:46.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:09:46.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:09:46.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:09:46.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:09:46.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:09:46.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:46.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:09:46.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:09:46.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:46.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:09:46.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:09:46.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:46.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:09:46.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:09:46.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:09:46.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:09:46.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:09:46.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:46.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.26765097, 2021-01-15T19:09:46.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:09:46.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:46.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:46.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:46.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:46.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:46.801 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:09:46.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:46.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:46.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:46.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:46.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:46.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:46.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:09:46.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:46.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:09:46.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:09:46.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:46.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:09:46.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:09:46.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:46.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:09:46.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:09:46.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:46.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:09:46.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:09:46.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:09:46.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:09:46.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:46.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:09:46.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:09:46.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:09:46.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:09:46.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:46.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:09:46.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:09:46.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:09:46.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:09:46.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:46.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:09:46.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:09:46.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:46.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:09:46.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:09:46.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:46.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:09:46.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:09:46.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:46.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:46.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:46.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:09:46.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:09:46.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.817 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:46.817 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:46.817 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:46.819 INFO:tasks.ceph.mon.a.smithi001.stderr:2021-01-15T19:09:46.601+0000 7f9995812700 -1 mon.a@0(leader).osd e118 definitely_dead 0 2021-01-15T19:09:46.819 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:46.818+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:46.820 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:46.920 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:46.919+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:46.987 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:46.987 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:46.987 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:46.987 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:46.996 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:47.037 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:47.036+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:47.122 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:47.121+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:47.160 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:47.161 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:47.161 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:47.170 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:47.223 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:47.222+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:47.278 INFO:tasks.rados.rados.0.smithi001.stdout:458: done (4 left) 2021-01-15T19:09:47.278 INFO:tasks.rados.rados.0.smithi001.stdout:459: done (3 left) 2021-01-15T19:09:47.278 INFO:tasks.rados.rados.0.smithi001.stdout:462: done (2 left) 2021-01-15T19:09:47.279 INFO:tasks.rados.rados.0.smithi001.stdout:463: done (1 left) 2021-01-15T19:09:47.279 INFO:tasks.rados.rados.0.smithi001.stdout:464: done (0 left) 2021-01-15T19:09:47.279 INFO:tasks.rados.rados.0.smithi001.stdout:465: setattr oid 50 current snap is 50 2021-01-15T19:09:47.280 INFO:tasks.rados.rados.0.smithi001.stdout:466: rmattr oid 9 current snap is 50 2021-01-15T19:09:47.280 INFO:tasks.rados.rados.0.smithi001.stdout:466: done (1 left) 2021-01-15T19:09:47.280 INFO:tasks.rados.rados.0.smithi001.stdout:467: read oid 40 snap -1 2021-01-15T19:09:47.280 INFO:tasks.rados.rados.0.smithi001.stdout:467: expect deleted 2021-01-15T19:09:47.280 INFO:tasks.rados.rados.0.smithi001.stdout:468: delete oid 15 current snap is 50 2021-01-15T19:09:47.292 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 125 (ObjNum 53 snap 1 seq_num 53) dirty exists 2021-01-15T19:09:47.292 INFO:teuthology.orchestra.run.smithi001.stderr:marked out osd.6. 2021-01-15T19:09:47.303 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:47.303 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:47.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:47.304 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:47.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:47.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:47.306 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.013877528999998, 2021-01-15T19:09:47.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:47.307 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.307 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.309 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:47.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:47.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:47.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:47.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:47.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:47.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:47.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:47.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:47.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:47.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:47.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:47.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:47.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:47.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:47.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:47.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:47.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:47.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:47.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:47.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:47.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:47.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:47.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:47.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:47.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:47.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:47.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:47.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:47.339 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.339 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:47.341 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:47.342 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.082114763000007, 2021-01-15T19:09:47.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:47.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.343 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.345 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:47.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.345 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:47.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:47.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:47.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:47.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:47.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:47.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:47.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:47.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:47.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:47.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:47.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:47.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:47.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:47.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:47.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:47.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:47.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:47.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:47.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:47.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:47.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:47.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:47.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:47.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:47.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:47.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:47.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:47.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.365 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.365 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.366 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:47.366 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:47.366 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.079826722000007, 2021-01-15T19:09:47.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:47.367 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.367 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:47.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:47.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:47.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:47.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:47.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:47.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:47.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:47.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:47.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:47.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:47.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:47.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:47.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:47.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:47.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:47.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:47.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:47.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:47.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:47.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:47.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:47.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:47.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.381 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.382 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:47.382 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:47.383 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.077237096999994, 2021-01-15T19:09:47.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:47.383 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.383 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.383 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.384 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:47.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.384 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:47.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:47.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:47.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:47.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:47.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:47.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:47.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:47.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:47.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:47.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:47.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:47.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:47.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:47.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:47.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:47.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:47.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:47.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:47.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:47.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:47.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:47.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:47.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:47.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:47.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.076328818999997, 2021-01-15T19:09:47.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:47.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:47.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:47.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:47.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:47.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:47.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:47.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:47.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.413 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.107644911999998, 2021-01-15T19:09:47.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:47.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:47.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:47.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:47.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:47.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:47.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:47.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:47.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:47.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:47.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:47.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:47.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:47.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.431 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:47.432 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.107495174999997, 2021-01-15T19:09:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.433 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:47.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:47.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:47.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:47.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:47.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.107343276999998, 2021-01-15T19:09:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:47.452 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.453 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:47.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:47.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:47.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:47.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:47.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:47.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:47.470 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.107217965000004, 2021-01-15T19:09:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:47.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:47.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:47.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:47.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:47.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:47.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:47.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:47.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:47.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:47.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:47.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:47.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:47.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.086385390000004, 2021-01-15T19:09:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:47.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:47.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:47.505 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.080092058000005, 2021-01-15T19:09:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.507 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:47.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:47.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:47.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:47.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:47.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:47.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:47.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:47.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:47.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:47.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:47.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:47.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:47.520 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.118032545000005, 2021-01-15T19:09:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.521 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:47.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:47.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:47.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:47.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:47.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.537 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:47.538 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.055660564999997, 2021-01-15T19:09:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.539 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:47.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:47.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:47.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.553 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.554 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.052358064000003, 2021-01-15T19:09:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.555 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:47.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:47.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:47.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:47.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:47.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:47.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:47.570 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.051036198000006, 2021-01-15T19:09:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.571 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.572 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:47.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:47.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:47.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:47.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:47.586 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.050665945000006, 2021-01-15T19:09:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:47.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:47.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:47.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:47.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.601 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:47.602 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.050331552000003, 2021-01-15T19:09:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.603 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:47.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:47.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:47.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:47.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:47.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:47.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:47.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.109070293000002, 2021-01-15T19:09:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.618 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:47.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:47.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:47.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:47.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:47.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:47.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:47.631 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.116707421000001, 2021-01-15T19:09:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:47.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.633 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:47.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:47.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:47.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:47.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:47.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:47.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:47.649 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.086713217000003, 2021-01-15T19:09:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:47.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:47.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:47.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:47.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:47.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:47.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:47.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:47.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:47.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:47.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:47.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:47.664 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:47.666 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:47.664+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:47.667 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:47.767 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:47.765+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:47.832 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:47.832 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:47.832 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:47.833 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:47.842 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:47.886 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:47.885+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:47.966 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:47.966 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:47.967 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:47.969 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:47.968+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:47.975 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:48.069 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:48.068+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:48.141 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:48.143 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:48.143 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:48.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:48.144 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:48.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.144 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:09:48.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:48.145 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.955916959999996, 2021-01-15T19:09:48.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:09:48.146 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.146 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.146 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:48.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:48.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:09:48.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:09:48.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:09:48.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:09:48.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:09:48.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:09:48.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:09:48.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:09:48.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:09:48.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:09:48.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:09:48.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:09:48.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:09:48.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:09:48.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:09:48.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:09:48.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:09:48.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.157 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.158 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:09:48.158 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:48.158 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.874959509999996, 2021-01-15T19:09:48.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:09:48.159 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.159 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:48.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:48.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:09:48.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:09:48.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:09:48.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:09:48.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:09:48.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:09:48.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:09:48.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:09:48.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:09:48.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:09:48.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:09:48.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:09:48.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:09:48.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:09:48.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:09:48.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:09:48.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:09:48.169 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.170 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:09:48.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:48.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.870548236000005, 2021-01-15T19:09:48.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:09:48.171 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.171 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:48.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:48.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:09:48.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:09:48.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:09:48.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:09:48.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:09:48.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:09:48.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:09:48.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:09:48.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:09:48.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:09:48.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:09:48.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:09:48.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:09:48.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:09:48.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:09:48.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:09:48.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:09:48.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.183 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.184 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:48.184 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:48.184 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.908428783000005, 2021-01-15T19:09:48.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:09:48.185 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.185 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:48.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.186 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:09:48.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.186 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:48.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:48.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:09:48.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:09:48.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:09:48.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:09:48.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:09:48.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:09:48.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:09:48.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:09:48.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:09:48.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:09:48.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:09:48.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:09:48.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:09:48.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:09:48.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:09:48.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:09:48.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:09:48.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:48.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:09:48.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:09:48.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:09:48.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:09:48.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.200 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.200 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:09:48.201 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:48.201 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.908280765000001, 2021-01-15T19:09:48.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:09:48.201 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.202 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.202 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:09:48.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.203 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:48.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:48.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:09:48.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:09:48.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:09:48.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:09:48.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:09:48.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:09:48.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:09:48.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:09:48.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:09:48.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:09:48.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:09:48.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:48.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:09:48.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:09:48.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:09:48.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:09:48.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:48.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:09:48.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:09:48.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:09:48.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:09:48.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:09:48.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:09:48.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:09:48.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:09:48.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.218 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.219 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=201, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:48.219 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:48.219 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.787845248000004, 2021-01-15T19:09:48.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999, 2021-01-15T19:09:48.220 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.220 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:48.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:48.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6137999999999998e-05 2021-01-15T19:09:48.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352839+0000", 2021-01-15T19:09:48.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2480000000000002e-06 2021-01-15T19:09:48.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352848+0000", 2021-01-15T19:09:48.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4456e-05 2021-01-15T19:09:48.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352863+0000", 2021-01-15T19:09:48.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.755e-06 2021-01-15T19:09:48.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352869+0000", 2021-01-15T19:09:48.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01049957 2021-01-15T19:09:48.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.363368+0000", 2021-01-15T19:09:48.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001221172 2021-01-15T19:09:48.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.364589+0000", 2021-01-15T19:09:48.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999 2021-01-15T19:09:48.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.229 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.229 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.230 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=202, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:48.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:48.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.787729966000001, 2021-01-15T19:09:48.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293, 2021-01-15T19:09:48.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.231 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:48.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:48.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1519e-05 2021-01-15T19:09:48.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352950+0000", 2021-01-15T19:09:48.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3610000000000001e-06 2021-01-15T19:09:48.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352958+0000", 2021-01-15T19:09:48.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4105e-05 2021-01-15T19:09:48.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352972+0000", 2021-01-15T19:09:48.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3709999999999999e-06 2021-01-15T19:09:48.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352978+0000", 2021-01-15T19:09:48.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013549736999999999 2021-01-15T19:09:48.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.366527+0000", 2021-01-15T19:09:48.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025352 2021-01-15T19:09:48.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.369063+0000", 2021-01-15T19:09:48.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293 2021-01-15T19:09:48.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.240 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.240 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=257, reqid=client.4418.0:863, at_version=85'126, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:48.240 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:48.241 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.768286433999997, 2021-01-15T19:09:48.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972, 2021-01-15T19:09:48.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:48.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:48.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000526152 2021-01-15T19:09:48.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372878+0000", 2021-01-15T19:09:48.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8391e-05 2021-01-15T19:09:48.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372906+0000", 2021-01-15T19:09:48.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2021-01-15T19:09:48.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372924+0000", 2021-01-15T19:09:48.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4420000000000001e-06 2021-01-15T19:09:48.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372931+0000", 2021-01-15T19:09:48.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040511399999999998 2021-01-15T19:09:48.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373336+0000", 2021-01-15T19:09:48.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4637000000000001e-05 2021-01-15T19:09:48.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373351+0000", 2021-01-15T19:09:48.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014127954 2021-01-15T19:09:48.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387479+0000", 2021-01-15T19:09:48.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010297 2021-01-15T19:09:48.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387582+0000", 2021-01-15T19:09:48.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972 2021-01-15T19:09:48.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=261, reqid=client.4418.0:849, at_version=85'130, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:48.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:48.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.741131594000002, 2021-01-15T19:09:48.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001, 2021-01-15T19:09:48.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.254 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:48.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:48.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041353340000000001 2021-01-15T19:09:48.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403642+0000", 2021-01-15T19:09:48.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6962999999999998e-05 2021-01-15T19:09:48.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403679+0000", 2021-01-15T19:09:48.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7178999999999999e-05 2021-01-15T19:09:48.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403696+0000", 2021-01-15T19:09:48.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:48.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403703+0000", 2021-01-15T19:09:48.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6183999999999997e-05 2021-01-15T19:09:48.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403749+0000", 2021-01-15T19:09:48.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5073e-05 2021-01-15T19:09:48.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403764+0000", 2021-01-15T19:09:48.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012428852000000001 2021-01-15T19:09:48.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416193+0000", 2021-01-15T19:09:48.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1882000000000002e-05 2021-01-15T19:09:48.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416245+0000", 2021-01-15T19:09:48.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001 2021-01-15T19:09:48.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.265 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.265 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.266 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:48.266 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:48.266 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.953959774000001, 2021-01-15T19:09:48.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:09:48.267 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.267 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.267 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.268 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:48.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:48.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:48.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:09:48.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:09:48.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:09:48.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:09:48.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:09:48.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:09:48.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:09:48.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:09:48.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:09:48.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:09:48.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:09:48.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:09:48.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:09:48.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:09:48.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:09:48.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:09:48.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:09:48.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:09:48.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:09:48.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.280 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:48.281 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:48.281 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.953768485000001, 2021-01-15T19:09:48.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:09:48.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.283 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:48.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.283 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:48.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:48.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:09:48.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:09:48.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:09:48.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:09:48.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:09:48.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:09:48.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:09:48.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:09:48.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:09:48.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:09:48.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:09:48.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:09:48.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:09:48.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:09:48.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:09:48.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:09:48.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:09:48.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:09:48.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:09:48.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:48.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:48.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.953606132000001, 2021-01-15T19:09:48.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:09:48.297 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.297 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.298 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.298 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:48.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.298 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:48.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:48.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:09:48.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:09:48.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:09:48.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:09:48.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:09:48.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:09:48.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:09:48.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:09:48.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:09:48.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:09:48.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:09:48.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:09:48.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:09:48.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:09:48.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:09:48.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:09:48.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:09:48.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:09:48.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:09:48.310 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.310 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.310 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.311 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:942 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~2203648,checksum] snapc 0=[] ondisk+read+known_if_redirected e90)", 2021-01-15T19:09:48.311 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:48.312 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.68487528, 2021-01-15T19:09:48.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817, 2021-01-15T19:09:48.312 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.312 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.313 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 942 2021-01-15T19:09:48.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.313 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:48.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:48.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3491000000000006e-05 2021-01-15T19:09:48.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455826+0000", 2021-01-15T19:09:48.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1099999999999997e-06 2021-01-15T19:09:48.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455834+0000", 2021-01-15T19:09:48.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2245e-05 2021-01-15T19:09:48.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455846+0000", 2021-01-15T19:09:48.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1660000000000002e-06 2021-01-15T19:09:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455851+0000", 2021-01-15T19:09:48.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001470865 2021-01-15T19:09:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457322+0000", 2021-01-15T19:09:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012032 2021-01-15T19:09:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457442+0000", 2021-01-15T19:09:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097176199999999997 2021-01-15T19:09:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.467160+0000", 2021-01-15T19:09:48.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817 2021-01-15T19:09:48.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.323 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.324 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s1 91/90 ECSubReadReply(tid=71, attrs_read=0))", 2021-01-15T19:09:48.324 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:48.324 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.924350884999999, 2021-01-15T19:09:48.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001, 2021-01-15T19:09:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032454459999999999 2021-01-15T19:09:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219533+0000", 2021-01-15T19:09:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2258e-05 2021-01-15T19:09:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219545+0000", 2021-01-15T19:09:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3614999999999999e-05 2021-01-15T19:09:48.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219559+0000", 2021-01-15T19:09:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2020000000000001e-06 2021-01-15T19:09:48.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219565+0000", 2021-01-15T19:09:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4172999999999998e-05 2021-01-15T19:09:48.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219619+0000", 2021-01-15T19:09:48.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098911899999999994 2021-01-15T19:09:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229510+0000", 2021-01-15T19:09:48.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001 2021-01-15T19:09:48.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.334 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 96/92 ECSubWrite(tid=404, reqid=osd.4.0:404, at_version=96'68, trim_to=0'0, roll_forward_to=93'66))", 2021-01-15T19:09:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.928498368, 2021-01-15T19:09:48.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125, 2021-01-15T19:09:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1780000000000002e-05 2021-01-15T19:09:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212192+0000", 2021-01-15T19:09:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2773e-05 2021-01-15T19:09:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212214+0000", 2021-01-15T19:09:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8345000000000001e-05 2021-01-15T19:09:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212233+0000", 2021-01-15T19:09:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8379999999999996e-06 2021-01-15T19:09:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212240+0000", 2021-01-15T19:09:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1579000000000002e-05 2021-01-15T19:09:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212291+0000", 2021-01-15T19:09:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3115999999999999e-05 2021-01-15T19:09:48.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212304+0000", 2021-01-15T19:09:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011237044 2021-01-15T19:09:48.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223541+0000", 2021-01-15T19:09:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9774999999999999e-05 2021-01-15T19:09:48.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223571+0000", 2021-01-15T19:09:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125 2021-01-15T19:09:48.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:982 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.921946982000001, 2021-01-15T19:09:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641, 2021-01-15T19:09:48.348 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.349 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 982 2021-01-15T19:09:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8164999999999999e-05 2021-01-15T19:09:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218719+0000", 2021-01-15T19:09:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6679999999999999e-06 2021-01-15T19:09:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218723+0000", 2021-01-15T19:09:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0639999999999999e-06 2021-01-15T19:09:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218729+0000", 2021-01-15T19:09:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2021-01-15T19:09:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218732+0000", 2021-01-15T19:09:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033278300000000002 2021-01-15T19:09:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219064+0000", 2021-01-15T19:09:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118377 2021-01-15T19:09:48.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219183+0000", 2021-01-15T19:09:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014915026 2021-01-15T19:09:48.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.234098+0000", 2021-01-15T19:09:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641 2021-01-15T19:09:48.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.921886219000001, 2021-01-15T19:09:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:09:48.361 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:09:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:48.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:09:48.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:09:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:09:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:09:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:09:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:09:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:09:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:09:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:09:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:09:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:09:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:09:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:09:48.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:09:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:09:48.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:48.374 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:48.374 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.921824529999999, 2021-01-15T19:09:48.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:09:48.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:48.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.375 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.375 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:09:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:09:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:09:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:09:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:09:48.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:09:48.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:09:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:09:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:09:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:09:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:09:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:09:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:09:48.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:09:48.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:09:48.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:09:48.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.386 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 110/52 ECSubWrite(tid=513, reqid=client.4418.0:1096, at_version=110'199, trim_to=0'0, roll_forward_to=109'197))", 2021-01-15T19:09:48.386 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9231639830000002, 2021-01-15T19:09:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999, 2021-01-15T19:09:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:48.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:48.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091685599999999996 2021-01-15T19:09:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218391+0000", 2021-01-15T19:09:48.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6659999999999998e-05 2021-01-15T19:09:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218428+0000", 2021-01-15T19:09:48.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6130999999999999e-05 2021-01-15T19:09:48.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218444+0000", 2021-01-15T19:09:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-05 2021-01-15T19:09:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218455+0000", 2021-01-15T19:09:48.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0050000000000003e-05 2021-01-15T19:09:48.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218515+0000", 2021-01-15T19:09:48.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4779e-05 2021-01-15T19:09:48.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218530+0000", 2021-01-15T19:09:48.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014431424 2021-01-15T19:09:48.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.232961+0000", 2021-01-15T19:09:48.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.893e-05 2021-01-15T19:09:48.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233020+0000", 2021-01-15T19:09:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999 2021-01-15T19:09:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1142 3.9s0 3:935d2ce1:::smithi00140323-50:head [create,setxattr nOg\\b]PYmQpZprhrNSoOiXch_OVhaejNkYNj\\rrb (1507) in=40b,setxattr WcdVgSb^Xpei^gpqgVORjX_ifYYlebi^lg_\\ (1997) in=36b,setxattr aQpVtT`^]OieemiqNUnaUfic[c]Wqmdis`XQ_Tqm (908) in=40b,setxattr nYb]ZVbStbOagQf^W[sgda\\nc]R]dSaefWh]adhg (1893) in=40b,setxattr U^ja\\c]ahh_WgcSQkePloVfUV[_dZZa_Qa^jZsgO (1086) in=40b,setxattr TUgWsbVkpRdhTUXqjX]eOikS`Wa]sVkaNpWTYbY[ (1399) in=40b,setxattr gXqmVqsjhpeU\\nn\\SoQ`SZfYeg^]omdk_pVTP`do (806) in=40b,setxattr TrVPgo]^oWNoalZnotgZbhqliYh[id^Q_loQnh^Z (1963) in=40b,setxattr _TS_[m^pOge]N^c\\[mgPssoXoqlONURs`Shj\\UYk (1647) in=40b,setxattr W^YiocPQgkgoftgn`q_YOW_UUYQpSd_s[^sOQfqh (874) in=40b,setxattr lbeYtWUYqpORgbP`aWmV\\TSo[odhaTnocqnTrlQr (945) in=40b,setxattr QdO^gYZc\\rqThdOUbdYlmofVstjYkpQs_ab]o_jT (1786) in=40b,setxattr hNj]sQWrnXhXNh[PPa`V^jVe_RrNYUN^^^\\VmZj[ (943) in=40b,setxattr s_fmbQYrifmdiN_RPaXpdaZd[\\WffOb^[VNTtQjW (383) in=40b,setxattr _kZU^ZanaPRW_XWjbYlgtXRWi^T[O^TVpkTs`npQ (1609) in=40b,setxattr qaNdNbjbVYZU[Ygrejofh`kRTiOS_jZ_PkYZ^U[p (733) in=40b,setxattr i_NbbQjhTiYWVccoc`[glilPmsbWr]RalgkhRmcc (417) in=40b,setxattr aZess_ORomNedc_mbO_QcQqabPbRt_aa\\dslZoPr (17332) in=40b,setxattr Roai^sie^qdaa[nObntZWmUWiV]flrp`qaaVRecS (1785) in=40b,setxattr ek^QaVgeZSSY_ck^tsPa^ncnge]eT_asrQ_NOa\\Q (1153) in=40b] snapc 5c=[5c,58,57,56,53,4b,44,41,26] ondisk+write+known_if_redirected e119)", 2021-01-15T19:09:48.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.86131259900000001, 2021-01-15T19:09:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875, 2021-01-15T19:09:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:48.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1142 2021-01-15T19:09:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.401 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:48.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4063999999999999e-05 2021-01-15T19:09:48.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:48.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279420+0000", 2021-01-15T19:09:48.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2021-01-15T19:09:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279428+0000", 2021-01-15T19:09:48.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3764999999999999e-05 2021-01-15T19:09:48.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:48.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279441+0000", 2021-01-15T19:09:48.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4460000000000004e-06 2021-01-15T19:09:48.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279449+0000", 2021-01-15T19:09:48.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5884000000000003e-05 2021-01-15T19:09:48.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:48.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279495+0000", 2021-01-15T19:09:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014788 2021-01-15T19:09:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:48.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279643+0000", 2021-01-15T19:09:48.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104872 2021-01-15T19:09:48.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:48.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.280748+0000", 2021-01-15T19:09:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044365569999999998 2021-01-15T19:09:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:48.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.285184+0000", 2021-01-15T19:09:48.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064052199999999997 2021-01-15T19:09:48.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291589+0000", 2021-01-15T19:09:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4213000000000002e-05 2021-01-15T19:09:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:48.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291664+0000", 2021-01-15T19:09:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875 2021-01-15T19:09:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:48.415 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:48.415 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:48.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:09:48.417 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:48.415+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:48.418 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:48.518 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:48.516+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:48.571 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1143 3.14s0 3.1c4d9eb4 (undecoded) ondisk+read+known_if_redirected e119)", 2021-01-15T19:09:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:09:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.290678083, 2021-01-15T19:09:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.290708425, 2021-01-15T19:09:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "delayed", 2021-01-15T19:09:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1143 2021-01-15T19:09:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:09:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:09:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4906999999999994e-05 2021-01-15T19:09:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279407+0000", 2021-01-15T19:09:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4280000000000002e-06 2021-01-15T19:09:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279416+0000", 2021-01-15T19:09:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5781e-05 2021-01-15T19:09:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279432+0000", 2021-01-15T19:09:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2230000000000007e-06 2021-01-15T19:09:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279440+0000", 2021-01-15T19:09:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033244699999999998 2021-01-15T19:09:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279773+0000", 2021-01-15T19:09:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4389000000000001e-05 2021-01-15T19:09:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279787+0000", 2021-01-15T19:09:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0057050219999999997 2021-01-15T19:09:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.285492+0000", 2021-01-15T19:09:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3919999999999996e-06 2021-01-15T19:09:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.285497+0000", 2021-01-15T19:09:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.90919155600000001 2021-01-15T19:09:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.194688+0000", 2021-01-15T19:09:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6659999999999999e-06 2021-01-15T19:09:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.194693+0000", 2021-01-15T19:09:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91535081100000004 2021-01-15T19:09:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1144 3.14s0 3.1c4d9eb4 (undecoded) ondisk+read+known_if_redirected e119)", 2021-01-15T19:09:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279492+0000", 2021-01-15T19:09:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.290528399, 2021-01-15T19:09:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.290674197, 2021-01-15T19:09:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "delayed", 2021-01-15T19:09:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1144 2021-01-15T19:09:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279492+0000", 2021-01-15T19:09:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279492+0000", 2021-01-15T19:09:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4564999999999997e-05 2021-01-15T19:09:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279546+0000", 2021-01-15T19:09:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8660000000000006e-06 2021-01-15T19:09:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279554+0000", 2021-01-15T19:09:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3695e-05 2021-01-15T19:09:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279568+0000", 2021-01-15T19:09:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2729999999999999e-06 2021-01-15T19:09:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279574+0000", 2021-01-15T19:09:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030637600000000002 2021-01-15T19:09:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279880+0000", 2021-01-15T19:09:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.383e-05 2021-01-15T19:09:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279894+0000", 2021-01-15T19:09:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0056273599999999997 2021-01-15T19:09:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.285522+0000", 2021-01-15T19:09:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5700000000000001e-07 2021-01-15T19:09:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.285522+0000", 2021-01-15T19:09:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.90920528899999997 2021-01-15T19:09:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.194728+0000", 2021-01-15T19:09:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8899999999999998e-07 2021-01-15T19:09:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.194729+0000", 2021-01-15T19:09:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91523710000000003 2021-01-15T19:09:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1145 3.14s0 3.1c4d9eb4 (undecoded) ondisk+read+known_if_redirected e119)", 2021-01-15T19:09:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279617+0000", 2021-01-15T19:09:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.2904030689999999, 2021-01-15T19:09:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906559609999999, 2021-01-15T19:09:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "delayed", 2021-01-15T19:09:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1145 2021-01-15T19:09:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279617+0000", 2021-01-15T19:09:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279617+0000", 2021-01-15T19:09:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6070999999999999e-05 2021-01-15T19:09:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279673+0000", 2021-01-15T19:09:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0979999999999996e-06 2021-01-15T19:09:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279680+0000", 2021-01-15T19:09:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2631e-05 2021-01-15T19:09:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279693+0000", 2021-01-15T19:09:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279698+0000", 2021-01-15T19:09:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227946 2021-01-15T19:09:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279926+0000", 2021-01-15T19:09:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0852e-05 2021-01-15T19:09:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279937+0000", 2021-01-15T19:09:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005609223 2021-01-15T19:09:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.285546+0000", 2021-01-15T19:09:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8599999999999996e-07 2021-01-15T19:09:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.285547+0000", 2021-01-15T19:09:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.90921026199999999 2021-01-15T19:09:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.194757+0000", 2021-01-15T19:09:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1900000000000002e-07 2021-01-15T19:09:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.194758+0000", 2021-01-15T19:09:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.915141129 2021-01-15T19:09:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s1 3.18aae252 (undecoded) ondisk+retry+write+peerstat_old+known_if_redirected e120)", 2021-01-15T19:09:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:48.197788+0000", 2021-01-15T19:09:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.37223254900000002, 2021-01-15T19:09:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.372593548, 2021-01-15T19:09:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "delayed", 2021-01-15T19:09:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:09:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.197788+0000", 2021-01-15T19:09:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.197788+0000", 2021-01-15T19:09:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00075354800000000004 2021-01-15T19:09:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.198541+0000", 2021-01-15T19:09:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0121e-05 2021-01-15T19:09:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.198551+0000", 2021-01-15T19:09:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7960000000000001e-05 2021-01-15T19:09:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.198569+0000", 2021-01-15T19:09:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3889999999999994e-06 2021-01-15T19:09:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.198579+0000", 2021-01-15T19:09:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121922 2021-01-15T19:09:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.198700+0000", 2021-01-15T19:09:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.681e-06 2021-01-15T19:09:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.198704+0000", 2021-01-15T19:09:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00091662099999999995 2021-01-15T19:09:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:09:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 4 2021-01-15T19:09:48.636 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:48.637 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:48.638 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:48.638 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:48.638 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:48.636+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:48.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:48.645 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:48.667 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:48.688 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:48.710 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:48.710 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.712 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:48.712 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:48.712 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:48.713 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:48.713 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:48.738 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:48.737+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:48.777 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:48.777 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:48.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:48.839 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:48.837+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:48.919 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:48.919 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:48.930 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:48.971 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:48.969+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:49.040 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:49.039+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:49.102 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:49.102 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:49.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:49.153 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:49.152+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:49.219 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:09:49.224 INFO:tasks.rados.rados.0.smithi001.stdout:465: done (2 left) 2021-01-15T19:09:49.225 INFO:tasks.rados.rados.0.smithi001.stdout:467: done (1 left) 2021-01-15T19:09:49.225 INFO:tasks.rados.rados.0.smithi001.stdout:468: done (0 left) 2021-01-15T19:09:49.225 INFO:tasks.rados.rados.0.smithi001.stdout:469: rmattr oid 5 current snap is 50 2021-01-15T19:09:49.225 INFO:tasks.rados.rados.0.smithi001.stdout:470: read oid 21 snap 43 2021-01-15T19:09:49.226 INFO:tasks.rados.rados.0.smithi001.stdout:470: expect (ObjNum 130 snap 31 seq_num 130) 2021-01-15T19:09:49.226 INFO:tasks.rados.rados.0.smithi001.stdout:471: rollback oid 50 current snap is 50 2021-01-15T19:09:49.226 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 50 to 49 2021-01-15T19:09:49.226 INFO:tasks.rados.rados.0.smithi001.stdout:472: read oid 42 snap -1 2021-01-15T19:09:49.226 INFO:tasks.rados.rados.0.smithi001.stdout:472: expect (ObjNum 41 snap 0 seq_num 41) 2021-01-15T19:09:49.227 INFO:tasks.rados.rados.0.smithi001.stdout:473: copy_from oid 3 from oid 39 current snap is 50 2021-01-15T19:09:49.231 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 10 current snap is 50 2021-01-15T19:09:49.231 INFO:tasks.rados.rados.0.smithi001.stdout:474: seq_num 172 ranges {2703360=483328} 2021-01-15T19:09:49.242 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:49.241+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:49.251 INFO:tasks.rados.rados.0.smithi001.stdout:474: writing smithi00140323-10 from 2703360 to 3186688 tid 1 2021-01-15T19:09:49.251 INFO:tasks.rados.rados.0.smithi001.stdout:475: snap_create 2021-01-15T19:09:49.251 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 122 (ObjNum 163 snap 43 seq_num 163) dirty exists 2021-01-15T19:09:49.251 INFO:tasks.rados.rados.0.smithi001.stdout:471: finishing rollback tid 0 to smithi00140323-50 2021-01-15T19:09:49.252 INFO:tasks.rados.rados.0.smithi001.stdout:471: finishing rollback tid 1 to smithi00140323-50 2021-01-15T19:09:49.252 INFO:tasks.rados.rados.0.smithi001.stdout:471: finishing rollback tid 2 to smithi00140323-50 2021-01-15T19:09:49.252 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 128 (ObjNum 53 snap 1 seq_num 53) dirty exists 2021-01-15T19:09:49.252 INFO:tasks.rados.rados.0.smithi001.stdout:472: expect (ObjNum 41 snap 0 seq_num 41) 2021-01-15T19:09:49.280 INFO:tasks.rados.rados.0.smithi001.stdout:470: expect (ObjNum 130 snap 31 seq_num 130) 2021-01-15T19:09:49.343 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:49.344 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:49.344 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:49.342+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:49.360 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:49.401 INFO:tasks.rados.rados.0.smithi001.stdout:473: finishing copy_from to smithi00140323-3 2021-01-15T19:09:49.401 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 53 (ObjNum 170 snap 49 seq_num 170) dirty exists 2021-01-15T19:09:49.401 INFO:tasks.rados.rados.0.smithi001.stdout:473: finishing copy_from racing read to smithi00140323-3 2021-01-15T19:09:49.401 INFO:tasks.rados.rados.0.smithi001.stdout:474: finishing write tid 1 to smithi00140323-10 2021-01-15T19:09:49.402 INFO:tasks.rados.rados.0.smithi001.stdout:474: finishing write tid 2 to smithi00140323-10 2021-01-15T19:09:49.402 INFO:tasks.rados.rados.0.smithi001.stdout:474: finishing write tid 3 to smithi00140323-10 2021-01-15T19:09:49.402 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 70 (ObjNum 172 snap 50 seq_num 172) dirty exists 2021-01-15T19:09:49.403 INFO:tasks.rados.rados.0.smithi001.stdout:474: left oid 10 (ObjNum 172 snap 50 seq_num 172) 2021-01-15T19:09:49.445 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:49.444+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:49.545 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:49.543+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:49.568 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:49.568 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:49.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:49.666 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:49.665+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:49.747 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:49.745+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:49.777 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:49.777 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:49.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:49.871 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:49.870+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:49.949 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:49.948+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:49.959 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:49.959 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:49.969 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:50.054 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:50.053+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:50.141 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:50.142 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:50.151 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:50.149+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:50.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:50.199 INFO:tasks.rados.rados.0.smithi001.stdout:469: done (6 left) 2021-01-15T19:09:50.199 INFO:tasks.rados.rados.0.smithi001.stdout:470: done (5 left) 2021-01-15T19:09:50.199 INFO:tasks.rados.rados.0.smithi001.stdout:471: done (4 left) 2021-01-15T19:09:50.200 INFO:tasks.rados.rados.0.smithi001.stdout:472: done (3 left) 2021-01-15T19:09:50.200 INFO:tasks.rados.rados.0.smithi001.stdout:473: done (2 left) 2021-01-15T19:09:50.200 INFO:tasks.rados.rados.0.smithi001.stdout:474: done (1 left) 2021-01-15T19:09:50.200 INFO:tasks.rados.rados.0.smithi001.stdout:475: done (0 left) 2021-01-15T19:09:50.201 INFO:tasks.rados.rados.0.smithi001.stdout:476: read oid 24 snap 45 2021-01-15T19:09:50.201 INFO:tasks.rados.rados.0.smithi001.stdout:476: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:50.201 INFO:tasks.rados.rados.0.smithi001.stdout:477: rollback oid 36 current snap is 51 2021-01-15T19:09:50.201 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 36 to 47 2021-01-15T19:09:50.202 INFO:tasks.rados.rados.0.smithi001.stdout:478: rollback oid 20 current snap is 51 2021-01-15T19:09:50.202 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 20 to 38 2021-01-15T19:09:50.202 INFO:tasks.rados.rados.0.smithi001.stdout:479: read oid 46 snap -1 2021-01-15T19:09:50.202 INFO:tasks.rados.rados.0.smithi001.stdout:479: expect (ObjNum 45 snap 0 seq_num 45) 2021-01-15T19:09:50.202 INFO:tasks.rados.rados.0.smithi001.stdout:480: snap_remove snap 50 2021-01-15T19:09:50.251 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:50.250+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:50.335 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:50.335 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:50.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:50.387 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:50.386+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:50.453 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:50.450+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:50.518 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:50.519 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:50.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:50.569 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:50.568+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:50.654 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:50.652+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:50.700 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:50.700 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:50.710 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:50.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:50.794 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:50.793+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:50.840 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:50.841 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:50.850 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:50.895 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:50.894+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:50.896 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:50.917 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:50.936 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:50.956 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:50.955+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:50.957 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:50.976 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:50.976 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:50.977 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:50.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:51.019 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:51.019 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.020 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.020 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.020 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.020 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.020 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:51.021 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:51.022 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:51.022 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:51.069 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:51.068+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:51.153 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:449 3.8s0 3:1009af39:::smithi00140323-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.885317733999997, 2021-01-15T19:09:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001, 2021-01-15T19:09:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 449 2021-01-15T19:09:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264542+0000", 2021-01-15T19:09:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9189e-05 2021-01-15T19:09:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264581+0000", 2021-01-15T19:09:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7649999999999993e-06 2021-01-15T19:09:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264589+0000", 2021-01-15T19:09:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4618e-05 2021-01-15T19:09:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264604+0000", 2021-01-15T19:09:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7940000000000008e-06 2021-01-15T19:09:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264612+0000", 2021-01-15T19:09:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6378999999999996e-05 2021-01-15T19:09:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264679+0000", 2021-01-15T19:09:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4648999999999996e-05 2021-01-15T19:09:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.264753+0000", 2021-01-15T19:09:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016473695 2021-01-15T19:09:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281227+0000", 2021-01-15T19:09:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2460000000000007e-05 2021-01-15T19:09:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.281309+0000", 2021-01-15T19:09:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044761740000000003 2021-01-15T19:09:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285786+0000", 2021-01-15T19:09:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8227999999999997e-05 2021-01-15T19:09:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285864+0000", 2021-01-15T19:09:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105399 2021-01-15T19:09:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.285969+0000", 2021-01-15T19:09:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021427350000000001 2021-01-15T19:09:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:694 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.968645232, 2021-01-15T19:09:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661, 2021-01-15T19:09:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 694 2021-01-15T19:09:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181214+0000", 2021-01-15T19:09:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328e-05 2021-01-15T19:09:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181242+0000", 2021-01-15T19:09:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7009999999999998e-06 2021-01-15T19:09:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181245+0000", 2021-01-15T19:09:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2859999999999996e-06 2021-01-15T19:09:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181252+0000", 2021-01-15T19:09:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2021-01-15T19:09:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181254+0000", 2021-01-15T19:09:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5302000000000005e-05 2021-01-15T19:09:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181330+0000", 2021-01-15T19:09:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2021-01-15T19:09:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181340+0000", 2021-01-15T19:09:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077712880000000003 2021-01-15T19:09:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189111+0000", 2021-01-15T19:09:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111661 2021-01-15T19:09:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.189223+0000", 2021-01-15T19:09:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014541979 2021-01-15T19:09:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.203765+0000", 2021-01-15T19:09:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022550661 2021-01-15T19:09:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:09:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.968581432000001, 2021-01-15T19:09:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:09:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:09:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:09:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:09:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:09:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:09:51.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:09:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:09:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:09:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:09:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:09:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:09:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:09:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:09:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:09:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:09:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:09:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:09:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:09:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:09:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:09:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:09:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.965998665999997, 2021-01-15T19:09:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:09:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:09:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:09:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:09:51.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:09:51.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:09:51.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:09:51.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:09:51.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:09:51.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:09:51.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:09:51.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:09:51.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:09:51.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:09:51.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:09:51.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:09:51.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:09:51.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:09:51.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.240 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:09:51.240 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:51.240 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.965835724999998, 2021-01-15T19:09:51.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:09:51.241 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.241 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.241 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.242 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.242 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:09:51.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.242 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:51.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:09:51.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:09:51.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:09:51.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:09:51.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:09:51.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:09:51.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:09:51.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:09:51.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:09:51.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:09:51.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:09:51.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:09:51.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:09:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:09:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:09:51.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:09:51.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.253 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.254 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:51.254 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:51.254 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.804793079, 2021-01-15T19:09:51.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:09:51.254 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.255 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.255 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.255 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:09:51.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.256 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:51.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:09:51.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:09:51.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:09:51.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:09:51.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:09:51.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:09:51.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:09:51.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:09:51.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:09:51.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:09:51.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:09:51.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:09:51.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:09:51.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:09:51.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:09:51.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:09:51.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:09:51.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:09:51.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:09:51.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:09:51.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:09:51.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:09:51.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:09:51.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:09:51.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:09:51.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:09:51.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.277 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.277 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.278 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:51.278 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:51.279 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.804685622999997, 2021-01-15T19:09:51.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:09:51.279 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.279 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.279 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.280 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:09:51.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:51.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:09:51.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:09:51.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:09:51.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:09:51.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:09:51.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:09:51.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:09:51.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:09:51.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:09:51.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:09:51.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:09:51.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:09:51.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:09:51.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:09:51.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:09:51.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:09:51.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:51.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:09:51.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:09:51.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:09:51.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:09:51.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:51.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:09:51.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:09:51.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:09:51.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:09:51.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:09:51.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:09:51.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:09:51.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:09:51.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.297 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.302 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:51.302 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:51.302 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.671618756999997, 2021-01-15T19:09:51.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:09:51.303 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.303 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.303 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.304 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.304 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.304 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:09:51.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.304 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:09:51.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:09:51.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:09:51.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:09:51.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:09:51.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:09:51.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:09:51.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:09:51.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:09:51.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:09:51.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:09:51.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:09:51.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:09:51.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:09:51.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:09:51.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:09:51.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:09:51.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:09:51.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:09:51.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:09:51.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:09:51.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:09:51.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:09:51.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:09:51.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:09:51.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:09:51.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.326 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.326 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.328 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:51.328 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:51.329 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.775651074000002, 2021-01-15T19:09:51.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:09:51.329 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.330 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:51.330 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.331 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.332 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:51.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.332 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:51.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:09:51.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:09:51.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:09:51.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:09:51.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:09:51.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:09:51.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:09:51.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:09:51.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:09:51.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:09:51.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:51.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:09:51.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:09:51.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:09:51.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:09:51.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:09:51.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:09:51.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:09:51.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.356 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.775520944, 2021-01-15T19:09:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:09:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.358 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:09:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:09:51.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:09:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:09:51.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:09:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:09:51.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:09:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:09:51.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:09:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:51.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:09:51.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:09:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:09:51.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:09:51.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:09:51.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:09:51.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:09:51.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:09:51.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.370 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.371 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:09:51.371 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:51.371 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.775435535, 2021-01-15T19:09:51.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:09:51.371 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.372 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:51.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.373 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:51.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.373 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:51.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:09:51.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:09:51.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:09:51.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:09:51.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:09:51.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:09:51.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:09:51.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:09:51.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:09:51.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:09:51.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:09:51.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:09:51.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:09:51.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:09:51.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:09:51.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:09:51.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:09:51.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:09:51.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.383 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:51.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:51.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.944261146999999, 2021-01-15T19:09:51.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:09:51.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.386 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:09:51.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:51.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:09:51.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:09:51.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:09:51.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:09:51.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:09:51.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:09:51.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:09:51.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:09:51.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:09:51.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:09:51.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:09:51.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:09:51.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:51.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:09:51.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:09:51.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:09:51.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:09:51.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:09:51.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:09:51.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:09:51.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:09:51.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:09:51.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:09:51.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:09:51.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:09:51.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:09:51.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:09:51.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:09:51.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:09:51.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:09:51.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:09:51.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.403 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.404 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:09:51.404 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:51.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.944172498, 2021-01-15T19:09:51.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:09:51.405 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.405 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.406 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:09:51.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.406 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:09:51.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:09:51.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:09:51.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:09:51.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:09:51.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:09:51.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:09:51.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:09:51.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:09:51.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:09:51.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:09:51.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:09:51.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:51.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:09:51.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:09:51.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:09:51.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:09:51.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:51.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:09:51.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:09:51.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:09:51.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:09:51.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:09:51.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:09:51.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:09:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:09:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:09:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:09:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:09:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:09:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:09:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:09:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:994 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.935098416999999, 2021-01-15T19:09:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998, 2021-01-15T19:09:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 994 2021-01-15T19:09:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214761+0000", 2021-01-15T19:09:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3285999999999999e-05 2021-01-15T19:09:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214805+0000", 2021-01-15T19:09:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-05 2021-01-15T19:09:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214818+0000", 2021-01-15T19:09:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5945000000000001e-05 2021-01-15T19:09:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214833+0000", 2021-01-15T19:09:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4690000000000002e-06 2021-01-15T19:09:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214840+0000", 2021-01-15T19:09:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9747999999999998e-05 2021-01-15T19:09:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214900+0000", 2021-01-15T19:09:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146553 2021-01-15T19:09:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215046+0000", 2021-01-15T19:09:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081407799999999998 2021-01-15T19:09:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215860+0000", 2021-01-15T19:09:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0095697720000000007 2021-01-15T19:09:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225430+0000", 2021-01-15T19:09:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012440451 2021-01-15T19:09:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237871+0000", 2021-01-15T19:09:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9866999999999998e-05 2021-01-15T19:09:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.237970+0000", 2021-01-15T19:09:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023209128999999998 2021-01-15T19:09:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:995 3.8s0 3:1009af39:::smithi00140323-25:head [rollback 3e] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.934978640000001, 2021-01-15T19:09:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001, 2021-01-15T19:09:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 995 2021-01-15T19:09:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214881+0000", 2021-01-15T19:09:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8172e-05 2021-01-15T19:09:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214919+0000", 2021-01-15T19:09:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4777999999999999e-05 2021-01-15T19:09:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214944+0000", 2021-01-15T19:09:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4625e-05 2021-01-15T19:09:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214959+0000", 2021-01-15T19:09:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2830000000000001e-06 2021-01-15T19:09:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.214967+0000", 2021-01-15T19:09:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002508929 2021-01-15T19:09:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217476+0000", 2021-01-15T19:09:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8955000000000006e-05 2021-01-15T19:09:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.217545+0000", 2021-01-15T19:09:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066629600000000003 2021-01-15T19:09:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218211+0000", 2021-01-15T19:09:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072214000000000002 2021-01-15T19:09:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225432+0000", 2021-01-15T19:09:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012976164 2021-01-15T19:09:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238409+0000", 2021-01-15T19:09:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3859000000000001e-05 2021-01-15T19:09:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238463+0000", 2021-01-15T19:09:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023581461000000001 2021-01-15T19:09:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:996 3.8s0 3:1009af39:::smithi00140323-25:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,19] ondisk+write+known_if_redirected e96)", 2021-01-15T19:09:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.934850154999999, 2021-01-15T19:09:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001, 2021-01-15T19:09:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 996 2021-01-15T19:09:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215010+0000", 2021-01-15T19:09:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9251000000000001e-05 2021-01-15T19:09:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215049+0000", 2021-01-15T19:09:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3900000000000004e-06 2021-01-15T19:09:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215056+0000", 2021-01-15T19:09:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6064000000000001e-05 2021-01-15T19:09:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215072+0000", 2021-01-15T19:09:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4790000000000004e-06 2021-01-15T19:09:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.215078+0000", 2021-01-15T19:09:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294249999999997 2021-01-15T19:09:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220107+0000", 2021-01-15T19:09:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4726000000000002e-05 2021-01-15T19:09:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220182+0000", 2021-01-15T19:09:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043078000000000002 2021-01-15T19:09:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220613+0000", 2021-01-15T19:09:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048209209999999997 2021-01-15T19:09:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.225434+0000", 2021-01-15T19:09:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013153299 2021-01-15T19:09:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238587+0000", 2021-01-15T19:09:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5006000000000002e-05 2021-01-15T19:09:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238642+0000", 2021-01-15T19:09:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023632341000000001 2021-01-15T19:09:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:09:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.960664439999999, 2021-01-15T19:09:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:09:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:09:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:09:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:09:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:09:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:09:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:09:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:09:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:09:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:09:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:09:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:09:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:09:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:09:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:09:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:09:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:09:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:09:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:09:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:09:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:09:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:09:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:09:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:09:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:09:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:09:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:09:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:09:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:09:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.960532364999999, 2021-01-15T19:09:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:09:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:09:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:09:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:09:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:09:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:09:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:09:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:09:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:09:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:09:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:09:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:09:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:09:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:09:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:09:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:09:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:09:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:09:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:09:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:09:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:09:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:09:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:09:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:09:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:09:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:09:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:09:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:09:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:09:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:09:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.957396848, 2021-01-15T19:09:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:09:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:09:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:09:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:09:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:09:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:09:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:09:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:09:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:09:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:09:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:09:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:09:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:09:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:09:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:09:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:09:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:09:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:09:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:09:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:09:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:09:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:09:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:09:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:09:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:09:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:09:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:09:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:09:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:09:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.957292821999999, 2021-01-15T19:09:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:09:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:09:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:09:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:09:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:09:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:09:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:09:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:09:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:09:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:09:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:09:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:09:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:09:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:09:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:09:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:09:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:09:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:09:51.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:09:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:09:51.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:09:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:09:51.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:09:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:09:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:09:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:09:51.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:09:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:09:51.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:09:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:09:51.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:51.541 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:51.541 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:51.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:09:51.544 INFO:tasks.rados.rados.0.smithi001.stdout:480: done (4 left) 2021-01-15T19:09:51.544 INFO:tasks.rados.rados.0.smithi001.stdout:481: snap_create 2021-01-15T19:09:51.544 INFO:tasks.rados.rados.0.smithi001.stdout:478: finishing rollback tid 0 to smithi00140323-20 2021-01-15T19:09:51.545 INFO:tasks.rados.rados.0.smithi001.stdout:478: finishing rollback tid 1 to smithi00140323-20 2021-01-15T19:09:51.545 INFO:tasks.rados.rados.0.smithi001.stdout:478: finishing rollback tid 2 to smithi00140323-20 2021-01-15T19:09:51.545 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 20 v 126 (ObjNum 19 snap 0 seq_num 19) dirty exists 2021-01-15T19:09:51.545 INFO:tasks.rados.rados.0.smithi001.stdout:477: finishing rollback tid 0 to smithi00140323-36 2021-01-15T19:09:51.545 INFO:tasks.rados.rados.0.smithi001.stdout:477: finishing rollback tid 1 to smithi00140323-36 2021-01-15T19:09:51.546 INFO:tasks.rados.rados.0.smithi001.stdout:477: finishing rollback tid 2 to smithi00140323-36 2021-01-15T19:09:51.546 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 126 (ObjNum 65 snap 8 seq_num 65) dirty exists 2021-01-15T19:09:51.546 INFO:tasks.rados.rados.0.smithi001.stdout:479: expect (ObjNum 45 snap 0 seq_num 45) 2021-01-15T19:09:51.546 INFO:tasks.rados.rados.0.smithi001.stdout:476: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:09:51.546 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:51.542+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:51.547 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:51.644 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:51.643+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:51.713 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:51.713 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:51.713 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:51.714 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:51.725 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:51.745 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:51.744+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:51.846 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:51.845+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:51.891 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:51.891 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:51.891 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:51.891 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:51.892 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:51.900 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:51.984 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:51.983+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:52.048 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:52.047+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:52.068 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:52.069 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:52.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:52.070 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:52.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.070 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:52.071 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:52.071 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.779171860999995, 2021-01-15T19:09:52.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:52.072 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.072 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.073 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.073 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.074 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.074 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:52.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.075 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:52.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:52.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:52.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:52.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:52.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:52.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:52.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:52.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:52.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:52.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:52.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:52.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:52.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:52.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:52.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:52.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:52.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:52.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:52.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:52.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:52.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:52.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:52.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:52.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:52.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:52.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:52.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:52.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.104 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.105 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:52.106 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:52.106 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.847409095000003, 2021-01-15T19:09:52.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:52.107 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.107 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.108 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:52.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.109 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:52.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:52.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:52.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:52.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:52.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:52.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:52.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:52.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:52.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:52.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:52.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:52.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:52.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:52.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:52.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:52.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:52.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:52.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:52.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:52.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:52.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:52.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:52.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:52.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:52.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:52.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:52.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:52.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.126 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:52.128 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:52.128 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.845121054000003, 2021-01-15T19:09:52.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:52.128 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.128 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.129 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:52.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.130 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:52.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:52.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:52.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:52.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:52.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:52.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:52.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:52.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:52.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:52.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:52.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:52.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:52.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:52.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:52.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:52.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:52.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:52.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:52.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:52.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:52.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:52.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:52.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.143 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.143 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:52.143 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:52.144 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.842531429000005, 2021-01-15T19:09:52.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:52.144 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.144 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.145 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.145 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:52.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.145 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:52.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:52.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:52.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:52.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:52.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:52.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:52.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:52.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:52.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:52.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:52.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:52.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:52.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:52.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:52.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:52.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:52.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:52.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:52.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:52.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:52.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:52.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:52.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.158 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.159 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:52.159 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:52.159 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.841623150999993, 2021-01-15T19:09:52.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:52.160 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.160 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.872939243999994, 2021-01-15T19:09:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.872789506999993, 2021-01-15T19:09:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.872637608999995, 2021-01-15T19:09:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.872512297, 2021-01-15T19:09:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.851679722, 2021-01-15T19:09:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.845386390000002, 2021-01-15T19:09:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.883326877000002, 2021-01-15T19:09:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:52.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:52.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:52.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:52.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:52.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:52.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:52.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:52.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:52.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:52.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:52.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:52.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:52.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:52.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:52.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:52.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:52.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:52.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:52.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.294 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.294 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.295 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:52.295 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:52.295 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.820954896999993, 2021-01-15T19:09:52.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:52.295 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.296 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:52.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:52.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:52.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:52.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:52.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:52.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:52.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:52.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:52.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:52.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:52.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:52.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:52.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:52.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:52.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:52.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:52.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:52.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:52.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:52.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:52.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:52.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.817652396, 2021-01-15T19:09:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:52.311 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.311 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.311 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.312 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.312 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:52.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.312 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:52.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:52.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:52.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:52.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:52.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:52.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:52.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:52.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:52.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:52.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:52.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:52.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:52.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:52.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:52.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:52.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:52.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:52.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:52.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:52.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:52.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:52.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:52.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.325 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:52.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:52.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.816330530000002, 2021-01-15T19:09:52.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:52.327 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.327 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.328 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:52.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.328 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:52.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:52.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:52.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:52.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:52.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:52.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:52.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:52.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:52.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:52.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:52.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:52.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:52.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:52.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:52.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:52.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:52.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:52.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:52.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:52.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:52.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:52.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:52.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.341 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.341 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.341 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:52.342 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:52.342 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.815960277000002, 2021-01-15T19:09:52.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:52.342 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.342 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.343 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.343 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:52.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.344 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:52.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:52.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:52.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:52.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:52.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:52.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:52.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:52.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:52.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:52.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:52.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:52.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:52.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:52.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:52.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:52.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:52.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:52.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:52.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:52.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:52.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:52.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.356 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.357 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:52.357 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:52.357 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.815625883999999, 2021-01-15T19:09:52.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:52.358 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.358 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.358 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.359 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:52.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.359 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:52.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:52.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:52.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:52.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:52.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:52.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:52.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:52.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:52.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:52.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:52.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:52.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:52.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:52.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:52.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:52.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:52.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:52.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:52.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:52.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:52.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.371 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.371 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:52.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:52.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.874364624999998, 2021-01-15T19:09:52.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:52.372 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.372 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.373 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:52.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:52.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:52.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:52.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:52.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:52.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:52.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:52.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:52.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:52.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:52.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:52.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:52.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:52.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:52.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:52.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:52.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:52.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:52.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:52.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:52.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:52.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:52.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:52.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.882001752999997, 2021-01-15T19:09:52.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:52.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.387 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.387 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:52.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:52.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:52.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:52.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:52.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:52.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:52.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:52.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:52.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:52.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:52.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:52.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:52.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:52.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:52.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:52.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:52.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:52.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:52.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:52.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:52.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:52.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:52.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:52.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:52.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:52.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:52.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:52.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:52.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:52.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:52.403 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.852007549, 2021-01-15T19:09:52.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:52.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:52.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:52.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:52.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:52.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:52.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:52.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:52.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:52.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:52.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:52.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:52.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:52.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:52.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:52.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:52.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:52.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:52.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:52.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:52.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:52.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:52.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:52.416 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.416 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.417 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.417 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:52.420 INFO:tasks.rados.rados.0.smithi001.stdout:476: done (4 left) 2021-01-15T19:09:52.420 INFO:tasks.rados.rados.0.smithi001.stdout:477: done (3 left) 2021-01-15T19:09:52.420 INFO:tasks.rados.rados.0.smithi001.stdout:478: done (2 left) 2021-01-15T19:09:52.420 INFO:tasks.rados.rados.0.smithi001.stdout:479: done (1 left) 2021-01-15T19:09:52.421 INFO:tasks.rados.rados.0.smithi001.stdout:481: done (0 left) 2021-01-15T19:09:52.421 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 7 current snap is 52 2021-01-15T19:09:52.421 INFO:tasks.rados.rados.0.smithi001.stdout:482: seq_num 173 ranges {4972544=499712} 2021-01-15T19:09:52.421 INFO:tasks.rados.rados.0.smithi001.stdout:482: writing smithi00140323-7 from 4972544 to 5472256 tid 1 2021-01-15T19:09:52.421 INFO:tasks.rados.rados.0.smithi001.stdout:483: delete oid 27 current snap is 52 2021-01-15T19:09:52.422 INFO:tasks.rados.rados.0.smithi001.stdout:483: done (1 left) 2021-01-15T19:09:52.422 INFO:tasks.rados.rados.0.smithi001.stdout:484: snap_create 2021-01-15T19:09:52.422 INFO:tasks.rados.rados.0.smithi001.stdout:482: finishing write tid 1 to smithi00140323-7 2021-01-15T19:09:52.422 INFO:tasks.rados.rados.0.smithi001.stdout:482: finishing write tid 2 to smithi00140323-7 2021-01-15T19:09:52.423 INFO:tasks.rados.rados.0.smithi001.stdout:482: finishing write tid 3 to smithi00140323-7 2021-01-15T19:09:52.423 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 166 (ObjNum 173 snap 52 seq_num 173) dirty exists 2021-01-15T19:09:52.423 INFO:tasks.rados.rados.0.smithi001.stdout:482: left oid 7 (ObjNum 173 snap 52 seq_num 173) 2021-01-15T19:09:52.423 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:09:52.424 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:52.418+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:52.424 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:52.520 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:52.518+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:52.586 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:52.586 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:52.586 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:52.595 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:52.620 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:52.619+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:52.667 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2021-01-15T19:09:52.667 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:52.667 INFO:tasks.thrashosds.thrasher:inject_pause on 7 2021-01-15T19:09:52.668 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2021-01-15T19:09:52.668 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2021-01-15T19:09:52.668 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2021-01-15T19:09:52.721 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:52.720+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:52.723 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:52.723 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:52.724 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:52.724 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:52.734 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:52.803 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:52.804 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2021-01-15T19:09:52.804 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:52.822 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:52.821+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:52.901 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:52.902 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:52.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:52.902 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:52.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.903 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:09:52.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:52.903 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.715135693999997, 2021-01-15T19:09:52.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:09:52.904 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.904 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:52.904 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:52.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:52.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:09:52.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:09:52.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:09:52.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:09:52.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:09:52.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:09:52.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:09:52.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:09:52.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:09:52.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:09:52.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:09:52.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:09:52.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:09:52.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:09:52.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:09:52.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:09:52.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:09:52.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.915 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.915 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:09:52.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:52.916 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.634178243999997, 2021-01-15T19:09:52.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:09:52.916 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.916 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:52.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:52.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:52.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:09:52.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:09:52.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:09:52.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:09:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:09:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:09:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:09:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:09:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:09:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:09:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:09:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:09:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:09:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:09:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:09:52.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:09:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:09:52.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.928 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:09:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.629766970000006, 2021-01-15T19:09:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:09:52.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:09:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:09:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:09:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:09:52.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:09:52.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:09:52.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:09:52.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:09:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:09:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:09:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:09:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:09:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:09:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:09:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:09:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:09:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:09:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:52.941 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.667647517000006, 2021-01-15T19:09:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:09:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:52.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:09:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:09:52.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:09:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:09:52.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:09:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:09:52.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:09:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:09:52.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:09:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:09:52.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:09:52.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:09:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:09:52.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:09:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:09:52.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:09:52.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:09:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:09:52.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:09:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:09:52.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:09:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:09:52.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:09:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.667499499000002, 2021-01-15T19:09:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:09:52.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:52.959 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:09:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:09:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:09:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:09:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:09:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:09:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:09:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:09:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:09:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:09:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:09:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:09:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:09:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:09:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:09:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:09:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:09:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:09:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:09:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:09:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:09:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:09:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:09:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:09:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.973 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=201, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:52.974 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.547063981999997, 2021-01-15T19:09:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999, 2021-01-15T19:09:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:52.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6137999999999998e-05 2021-01-15T19:09:52.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352839+0000", 2021-01-15T19:09:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2480000000000002e-06 2021-01-15T19:09:52.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352848+0000", 2021-01-15T19:09:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4456e-05 2021-01-15T19:09:52.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352863+0000", 2021-01-15T19:09:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.755e-06 2021-01-15T19:09:52.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352869+0000", 2021-01-15T19:09:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01049957 2021-01-15T19:09:52.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.363368+0000", 2021-01-15T19:09:52.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001221172 2021-01-15T19:09:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.364589+0000", 2021-01-15T19:09:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999 2021-01-15T19:09:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=202, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.546948700000002, 2021-01-15T19:09:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293, 2021-01-15T19:09:52.985 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1519e-05 2021-01-15T19:09:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352950+0000", 2021-01-15T19:09:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3610000000000001e-06 2021-01-15T19:09:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:52.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352958+0000", 2021-01-15T19:09:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4105e-05 2021-01-15T19:09:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352972+0000", 2021-01-15T19:09:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3709999999999999e-06 2021-01-15T19:09:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352978+0000", 2021-01-15T19:09:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013549736999999999 2021-01-15T19:09:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.366527+0000", 2021-01-15T19:09:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025352 2021-01-15T19:09:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.369063+0000", 2021-01-15T19:09:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293 2021-01-15T19:09:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=257, reqid=client.4418.0:863, at_version=85'126, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:52.995 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.527505167999998, 2021-01-15T19:09:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972, 2021-01-15T19:09:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:52.996 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:52.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000526152 2021-01-15T19:09:52.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372878+0000", 2021-01-15T19:09:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8391e-05 2021-01-15T19:09:52.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372906+0000", 2021-01-15T19:09:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2021-01-15T19:09:53.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372924+0000", 2021-01-15T19:09:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4420000000000001e-06 2021-01-15T19:09:53.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372931+0000", 2021-01-15T19:09:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040511399999999998 2021-01-15T19:09:53.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373336+0000", 2021-01-15T19:09:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4637000000000001e-05 2021-01-15T19:09:53.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373351+0000", 2021-01-15T19:09:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014127954 2021-01-15T19:09:53.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387479+0000", 2021-01-15T19:09:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010297 2021-01-15T19:09:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387582+0000", 2021-01-15T19:09:53.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972 2021-01-15T19:09:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=261, reqid=client.4418.0:849, at_version=85'130, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.500350328000003, 2021-01-15T19:09:53.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001, 2021-01-15T19:09:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041353340000000001 2021-01-15T19:09:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403642+0000", 2021-01-15T19:09:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6962999999999998e-05 2021-01-15T19:09:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403679+0000", 2021-01-15T19:09:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7178999999999999e-05 2021-01-15T19:09:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403696+0000", 2021-01-15T19:09:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403703+0000", 2021-01-15T19:09:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6183999999999997e-05 2021-01-15T19:09:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403749+0000", 2021-01-15T19:09:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5073e-05 2021-01-15T19:09:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403764+0000", 2021-01-15T19:09:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012428852000000001 2021-01-15T19:09:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416193+0000", 2021-01-15T19:09:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1882000000000002e-05 2021-01-15T19:09:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416245+0000", 2021-01-15T19:09:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001 2021-01-15T19:09:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:53.020 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.713178507999999, 2021-01-15T19:09:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:09:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.022 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:53.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:53.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:09:53.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:09:53.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:09:53.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:09:53.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:09:53.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:09:53.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:09:53.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:09:53.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:09:53.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:09:53.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:09:53.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:53.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:09:53.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:09:53.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:09:53.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:09:53.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:09:53.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:09:53.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:09:53.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:09:53.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.035 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:53.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:53.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.712987218999999, 2021-01-15T19:09:53.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:09:53.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.036 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.036 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.036 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.036 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.037 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:53.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.037 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:53.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:53.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:09:53.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:09:53.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:09:53.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:09:53.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:09:53.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:09:53.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:09:53.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:09:53.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:09:53.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:09:53.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:09:53.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:53.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:09:53.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:09:53.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:09:53.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:09:53.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:09:53.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:09:53.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:09:53.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:09:53.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:53.049 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:53.050 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.712824865999998, 2021-01-15T19:09:53.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:09:53.050 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.050 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.051 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.051 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.051 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:53.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.051 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:53.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:53.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:09:53.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:09:53.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:09:53.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:09:53.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:09:53.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:09:53.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:09:53.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:09:53.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:09:53.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:09:53.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:09:53.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:53.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:09:53.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:09:53.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:09:53.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:09:53.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:09:53.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:09:53.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:09:53.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:09:53.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.064 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:942 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~2203648,checksum] snapc 0=[] ondisk+read+known_if_redirected e90)", 2021-01-15T19:09:53.064 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:53.064 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.444094014000001, 2021-01-15T19:09:53.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817, 2021-01-15T19:09:53.065 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.065 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.066 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 942 2021-01-15T19:09:53.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:53.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:53.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3491000000000006e-05 2021-01-15T19:09:53.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455826+0000", 2021-01-15T19:09:53.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1099999999999997e-06 2021-01-15T19:09:53.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455834+0000", 2021-01-15T19:09:53.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2245e-05 2021-01-15T19:09:53.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455846+0000", 2021-01-15T19:09:53.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1660000000000002e-06 2021-01-15T19:09:53.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455851+0000", 2021-01-15T19:09:53.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001470865 2021-01-15T19:09:53.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457322+0000", 2021-01-15T19:09:53.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012032 2021-01-15T19:09:53.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457442+0000", 2021-01-15T19:09:53.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097176199999999997 2021-01-15T19:09:53.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.467160+0000", 2021-01-15T19:09:53.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817 2021-01-15T19:09:53.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.080 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.081 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s1 91/90 ECSubReadReply(tid=71, attrs_read=0))", 2021-01-15T19:09:53.081 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:53.081 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.683569618999996, 2021-01-15T19:09:53.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001, 2021-01-15T19:09:53.082 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.082 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:53.082 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032454459999999999 2021-01-15T19:09:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219533+0000", 2021-01-15T19:09:53.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2258e-05 2021-01-15T19:09:53.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219545+0000", 2021-01-15T19:09:53.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3614999999999999e-05 2021-01-15T19:09:53.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219559+0000", 2021-01-15T19:09:53.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2020000000000001e-06 2021-01-15T19:09:53.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219565+0000", 2021-01-15T19:09:53.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4172999999999998e-05 2021-01-15T19:09:53.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219619+0000", 2021-01-15T19:09:53.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098911899999999994 2021-01-15T19:09:53.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229510+0000", 2021-01-15T19:09:53.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001 2021-01-15T19:09:53.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.090 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.091 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 96/92 ECSubWrite(tid=404, reqid=osd.4.0:404, at_version=96'68, trim_to=0'0, roll_forward_to=93'66))", 2021-01-15T19:09:53.091 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:53.092 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.687717102000001, 2021-01-15T19:09:53.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125, 2021-01-15T19:09:53.092 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.092 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:53.092 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:53.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:53.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1780000000000002e-05 2021-01-15T19:09:53.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212192+0000", 2021-01-15T19:09:53.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2773e-05 2021-01-15T19:09:53.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212214+0000", 2021-01-15T19:09:53.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8345000000000001e-05 2021-01-15T19:09:53.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212233+0000", 2021-01-15T19:09:53.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8379999999999996e-06 2021-01-15T19:09:53.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212240+0000", 2021-01-15T19:09:53.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1579000000000002e-05 2021-01-15T19:09:53.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212291+0000", 2021-01-15T19:09:53.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3115999999999999e-05 2021-01-15T19:09:53.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:53.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212304+0000", 2021-01-15T19:09:53.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011237044 2021-01-15T19:09:53.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:53.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223541+0000", 2021-01-15T19:09:53.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9774999999999999e-05 2021-01-15T19:09:53.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223571+0000", 2021-01-15T19:09:53.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125 2021-01-15T19:09:53.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.103 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.104 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:982 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:53.104 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:53.104 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.681165715999999, 2021-01-15T19:09:53.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641, 2021-01-15T19:09:53.105 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.105 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.105 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.106 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 982 2021-01-15T19:09:53.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.106 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:53.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:53.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8164999999999999e-05 2021-01-15T19:09:53.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218719+0000", 2021-01-15T19:09:53.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6679999999999999e-06 2021-01-15T19:09:53.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218723+0000", 2021-01-15T19:09:53.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0639999999999999e-06 2021-01-15T19:09:53.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218729+0000", 2021-01-15T19:09:53.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2021-01-15T19:09:53.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218732+0000", 2021-01-15T19:09:53.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033278300000000002 2021-01-15T19:09:53.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219064+0000", 2021-01-15T19:09:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118377 2021-01-15T19:09:53.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219183+0000", 2021-01-15T19:09:53.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014915026 2021-01-15T19:09:53.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.234098+0000", 2021-01-15T19:09:53.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641 2021-01-15T19:09:53.115 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.116 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.116 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:53.117 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:53.117 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.681104952999998, 2021-01-15T19:09:53.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:09:53.117 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.117 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.118 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.118 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:09:53.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.119 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:53.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:53.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:09:53.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:09:53.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:09:53.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:09:53.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:09:53.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:09:53.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:09:53.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:09:53.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:09:53.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:09:53.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:09:53.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:09:53.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:09:53.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:09:53.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:09:53.128 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.128 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.128 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.129 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:53.129 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:53.129 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.681043263999999, 2021-01-15T19:09:53.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:09:53.130 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.130 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:53.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.130 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.131 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.131 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:09:53.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.131 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:53.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:53.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:09:53.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:09:53.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:09:53.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:09:53.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:09:53.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:09:53.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:09:53.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:09:53.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:09:53.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:09:53.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:09:53.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:09:53.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:09:53.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:09:53.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:09:53.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.141 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.141 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 110/52 ECSubWrite(tid=513, reqid=client.4418.0:1096, at_version=110'199, trim_to=0'0, roll_forward_to=109'197))", 2021-01-15T19:09:53.142 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:53.142 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.682382716999999, 2021-01-15T19:09:53.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999, 2021-01-15T19:09:53.142 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.142 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:53.143 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:53.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:53.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091685599999999996 2021-01-15T19:09:53.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218391+0000", 2021-01-15T19:09:53.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6659999999999998e-05 2021-01-15T19:09:53.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218428+0000", 2021-01-15T19:09:53.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6130999999999999e-05 2021-01-15T19:09:53.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218444+0000", 2021-01-15T19:09:53.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-05 2021-01-15T19:09:53.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218455+0000", 2021-01-15T19:09:53.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0050000000000003e-05 2021-01-15T19:09:53.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218515+0000", 2021-01-15T19:09:53.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4779e-05 2021-01-15T19:09:53.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:53.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218530+0000", 2021-01-15T19:09:53.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014431424 2021-01-15T19:09:53.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:53.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.232961+0000", 2021-01-15T19:09:53.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.893e-05 2021-01-15T19:09:53.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233020+0000", 2021-01-15T19:09:53.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999 2021-01-15T19:09:53.154 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.154 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.154 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.155 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1142 3.9s0 3:935d2ce1:::smithi00140323-50:head [create,setxattr nOg\\b]PYmQpZprhrNSoOiXch_OVhaejNkYNj\\rrb (1507) in=40b,setxattr WcdVgSb^Xpei^gpqgVORjX_ifYYlebi^lg_\\ (1997) in=36b,setxattr aQpVtT`^]OieemiqNUnaUfic[c]Wqmdis`XQ_Tqm (908) in=40b,setxattr nYb]ZVbStbOagQf^W[sgda\\nc]R]dSaefWh]adhg (1893) in=40b,setxattr U^ja\\c]ahh_WgcSQkePloVfUV[_dZZa_Qa^jZsgO (1086) in=40b,setxattr TUgWsbVkpRdhTUXqjX]eOikS`Wa]sVkaNpWTYbY[ (1399) in=40b,setxattr gXqmVqsjhpeU\\nn\\SoQ`SZfYeg^]omdk_pVTP`do (806) in=40b,setxattr TrVPgo]^oWNoalZnotgZbhqliYh[id^Q_loQnh^Z (1963) in=40b,setxattr _TS_[m^pOge]N^c\\[mgPssoXoqlONURs`Shj\\UYk (1647) in=40b,setxattr W^YiocPQgkgoftgn`q_YOW_UUYQpSd_s[^sOQfqh (874) in=40b,setxattr lbeYtWUYqpORgbP`aWmV\\TSo[odhaTnocqnTrlQr (945) in=40b,setxattr QdO^gYZc\\rqThdOUbdYlmofVstjYkpQs_ab]o_jT (1786) in=40b,setxattr hNj]sQWrnXhXNh[PPa`V^jVe_RrNYUN^^^\\VmZj[ (943) in=40b,setxattr s_fmbQYrifmdiN_RPaXpdaZd[\\WffOb^[VNTtQjW (383) in=40b,setxattr _kZU^ZanaPRW_XWjbYlgtXRWi^T[O^TVpkTs`npQ (1609) in=40b,setxattr qaNdNbjbVYZU[Ygrejofh`kRTiOS_jZ_PkYZ^U[p (733) in=40b,setxattr i_NbbQjhTiYWVccoc`[glilPmsbWr]RalgkhRmcc (417) in=40b,setxattr aZess_ORomNedc_mbO_QcQqabPbRt_aa\\dslZoPr (17332) in=40b,setxattr Roai^sie^qdaa[nObntZWmUWiV]flrp`qaaVRecS (1785) in=40b,setxattr ek^QaVgeZSSY_ck^tsPa^ncnge]eT_asrQ_NOa\\Q (1153) in=40b] snapc 5c=[5c,58,57,56,53,4b,44,41,26] ondisk+write+known_if_redirected e119)", 2021-01-15T19:09:53.155 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:53.156 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6205313329999997, 2021-01-15T19:09:53.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875, 2021-01-15T19:09:53.156 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:53.156 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:53.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:53.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:53.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:53.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1142 2021-01-15T19:09:53.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.157 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:53.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:53.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:53.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:53.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:53.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:53.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4063999999999999e-05 2021-01-15T19:09:53.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:53.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279420+0000", 2021-01-15T19:09:53.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2021-01-15T19:09:53.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:53.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279428+0000", 2021-01-15T19:09:53.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3764999999999999e-05 2021-01-15T19:09:53.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:53.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279441+0000", 2021-01-15T19:09:53.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4460000000000004e-06 2021-01-15T19:09:53.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:53.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279449+0000", 2021-01-15T19:09:53.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5884000000000003e-05 2021-01-15T19:09:53.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:53.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279495+0000", 2021-01-15T19:09:53.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014788 2021-01-15T19:09:53.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:53.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279643+0000", 2021-01-15T19:09:53.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104872 2021-01-15T19:09:53.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:53.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.280748+0000", 2021-01-15T19:09:53.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044365569999999998 2021-01-15T19:09:53.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:53.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.285184+0000", 2021-01-15T19:09:53.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064052199999999997 2021-01-15T19:09:53.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:53.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291589+0000", 2021-01-15T19:09:53.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4213000000000002e-05 2021-01-15T19:09:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291664+0000", 2021-01-15T19:09:53.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875 2021-01-15T19:09:53.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:53.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:53.171 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:53.171 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:53.173 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:53.171+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:53.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:53.218 INFO:tasks.rados.rados.0.smithi001.stdout:482: done (1 left) 2021-01-15T19:09:53.219 INFO:tasks.rados.rados.0.smithi001.stdout:484: done (0 left) 2021-01-15T19:09:53.219 INFO:tasks.rados.rados.0.smithi001.stdout:485: snap_remove snap 43 2021-01-15T19:09:53.273 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:53.272+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:53.343 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:53.343 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:53.344 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:53.344 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:53.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:53.362 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:53.373 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:53.372+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:53.383 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:53.405 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:53.426 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:53.447 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:53.468 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:53.475 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:53.473+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:53.490 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:53.496 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:53.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:53.498 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:53.499 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:53.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:53.576 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:53.575+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:53.678 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:53.676+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:53.680 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:53.680 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:53.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:53.782 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:53.781+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:53.867 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:53.867 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:53.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:53.879 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:53.878+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:53.980 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:53.979+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:54.010 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:54.010 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:54.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:54.082 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:54.080+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:54.154 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:54.155 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:54.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:54.183 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:54.182+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:54.220 INFO:tasks.rados.rados.0.smithi001.stdout:485: done (0 left) 2021-01-15T19:09:54.220 INFO:tasks.rados.rados.0.smithi001.stdout:486: snap_create 2021-01-15T19:09:54.283 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:54.282+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:54.350 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:54.351 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:54.361 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:54.401 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:54.400+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:54.424 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:09:54.485 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:54.484+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:54.534 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:54.535 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:54.546 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:54.586 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:54.585+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:54.681 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:54.681 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:54.687 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:54.686+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:54.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:54.788 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:54.787+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:54.868 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:54.868 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:54.878 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:54.924 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:54.922+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:54.990 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:54.989+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:55.010 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:55.010 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:55.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:55.091 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:55.090+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:55.191 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:55.192 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:55.190+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:55.201 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:55.223 INFO:tasks.rados.rados.0.smithi001.stdout:486: done (0 left) 2021-01-15T19:09:55.223 INFO:tasks.rados.rados.0.smithi001.stdout:487: snap_create 2021-01-15T19:09:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:09:55.292 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:55.291+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:55.340 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:55.340 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:55.351 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:55.392 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:55.391+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:55.493 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:55.492+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:55.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:09:55.514 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:55.515 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:55.524 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:55.594 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:55.592+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:55.648 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:55.649 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:55.653 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:55.658 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:09:55.673 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:55.694 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:55.696 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:55.693+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:55.713 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:55.734 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:55.755 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:55.782 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:55.783 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.783 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.783 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.783 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.784 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.784 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.784 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:55.784 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:55.784 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:09:55.785 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:55.786 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:55.786 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:55.786 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:55.786 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:55.787 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:55.794 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:09:55.796 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:09:55.795+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:09:55.897 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:55.896+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:55.918 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:55.919 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:55.919 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:55.919 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:55.920 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:55.929 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:09:56.014 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:56.013+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:56.097 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:56.097 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:56.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:56.098 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:56.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.098 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:09:56.098 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:56.099 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.808512277999995, 2021-01-15T19:09:56.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:09:56.100 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.100 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.100 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.101 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.101 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:09:56.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:56.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:09:56.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:09:56.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:09:56.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:09:56.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:09:56.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:09:56.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:09:56.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:09:56.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:09:56.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:09:56.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:09:56.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:09:56.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:56.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:09:56.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:09:56.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:09:56.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:09:56.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:09:56.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:09:56.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:09:56.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:09:56.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:09:56.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:09:56.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:09:56.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:09:56.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:09:56.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:09:56.117 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.117 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.118 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:56.118 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:56.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.876749512000004, 2021-01-15T19:09:56.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:09:56.119 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.119 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.120 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.121 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:09:56.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.121 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:56.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:09:56.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:09:56.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:09:56.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:09:56.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:09:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:09:56.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:09:56.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:09:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:09:56.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:09:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:09:56.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:09:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:56.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:09:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:09:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:09:56.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:09:56.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:09:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:09:56.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:09:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:09:56.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:09:56.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:09:56.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:09:56.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:09:56.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:09:56.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:09:56.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.137 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:56.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:56.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.874461471000004, 2021-01-15T19:09:56.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:09:56.138 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:09:56.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:56.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:09:56.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:09:56.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:09:56.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:09:56.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:09:56.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:09:56.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:09:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:09:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:09:56.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:09:56.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:09:56.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:09:56.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:09:56.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:09:56.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:09:56.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:09:56.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:09:56.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:09:56.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:09:56.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:09:56.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:09:56.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:09:56.154 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.155 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.155 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:09:56.156 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:56.156 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.871871846000005, 2021-01-15T19:09:56.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:09:56.156 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.156 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.157 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:09:56.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.158 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:56.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:09:56.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:09:56.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:09:56.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:09:56.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:09:56.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:09:56.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:09:56.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:09:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:09:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:09:56.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:09:56.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:09:56.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:09:56.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:09:56.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:09:56.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:09:56.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:09:56.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:09:56.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:09:56.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:09:56.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:09:56.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:09:56.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:09:56.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:56.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.870963567999993, 2021-01-15T19:09:56.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:09:56.173 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.173 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.174 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.174 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:09:56.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.174 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:56.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:09:56.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:09:56.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:09:56.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:09:56.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:09:56.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:09:56.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:09:56.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:09:56.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:09:56.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:09:56.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:09:56.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:09:56.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:09:56.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:09:56.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:09:56.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:09:56.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:09:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:09:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:09:56.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:09:56.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.186 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:56.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:56.188 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.902279660999994, 2021-01-15T19:09:56.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:09:56.188 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.190 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:09:56.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:56.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:09:56.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:09:56.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:09:56.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:09:56.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:09:56.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:09:56.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:09:56.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:09:56.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:09:56.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:09:56.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:09:56.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:09:56.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:56.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:09:56.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:09:56.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:09:56.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:09:56.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:09:56.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:09:56.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:09:56.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:09:56.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:09:56.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:09:56.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:09:56.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:09:56.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:09:56.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:09:56.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.206 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:56.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:56.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.902129923999993, 2021-01-15T19:09:56.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:09:56.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.208 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:09:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:09:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:09:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:09:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:09:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:09:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:09:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:09:56.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:09:56.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:09:56.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:09:56.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:09:56.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:09:56.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:56.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:09:56.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:09:56.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:09:56.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:09:56.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:56.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:09:56.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:09:56.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:09:56.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:09:56.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:09:56.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:09:56.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:09:56.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:09:56.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:09:56.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:09:56.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:09:56.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:09:56.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.225 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.226 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:09:56.226 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:56.226 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.901978025999995, 2021-01-15T19:09:56.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:09:56.227 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.227 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.227 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.228 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:09:56.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.228 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:56.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:09:56.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:09:56.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:09:56.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:09:56.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:09:56.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:09:56.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:09:56.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:09:56.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:09:56.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:09:56.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:09:56.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:09:56.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:56.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:09:56.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:09:56.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:09:56.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:09:56.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:09:56.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:09:56.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:09:56.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:09:56.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:09:56.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:09:56.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:09:56.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:09:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:09:56.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:09:56.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.243 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:09:56.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:56.244 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.901852714, 2021-01-15T19:09:56.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:09:56.244 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.245 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:09:56.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:56.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:09:56.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:09:56.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:09:56.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:09:56.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:09:56.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:09:56.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:09:56.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:09:56.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:09:56.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:09:56.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:09:56.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:09:56.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:09:56.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:09:56.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:09:56.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:09:56.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:09:56.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:56.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:09:56.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:09:56.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:09:56.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:09:56.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:09:56.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:09:56.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:09:56.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:09:56.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:09:56.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:09:56.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:09:56.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:09:56.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:09:56.264 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:56.264 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.881020139, 2021-01-15T19:09:56.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:09:56.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.265 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.265 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.266 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.266 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:09:56.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.267 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:56.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:09:56.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:09:56.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:09:56.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:09:56.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:09:56.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:09:56.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:09:56.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:09:56.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:09:56.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:09:56.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:09:56.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:09:56.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:09:56.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:09:56.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:09:56.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:09:56.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:09:56.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:09:56.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:09:56.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:09:56.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:09:56.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:09:56.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.280 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.281 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:09:56.281 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:56.281 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.874726807000002, 2021-01-15T19:09:56.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:09:56.282 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.282 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.282 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.283 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:09:56.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.283 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:56.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:09:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:09:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:09:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:09:56.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:09:56.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:09:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:09:56.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:09:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:09:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:09:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:09:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:09:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:09:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:09:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:09:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:09:56.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:09:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:09:56.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:09:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:09:56.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:09:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.912667294000002, 2021-01-15T19:09:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:09:56.298 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.299 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:09:56.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:56.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:09:56.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:09:56.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:09:56.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:09:56.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:09:56.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:09:56.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:09:56.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:09:56.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:09:56.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:09:56.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:09:56.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:09:56.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:09:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:09:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:09:56.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:09:56.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:09:56.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:09:56.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:09:56.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:09:56.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:09:56.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:09:56.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:09:56.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:09:56.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:09:56.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:09:56.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.315 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.316 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:56.316 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:56.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.850295313999993, 2021-01-15T19:09:56.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:09:56.317 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.317 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.318 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:09:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:56.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:09:56.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:09:56.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:09:56.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:09:56.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:09:56.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:09:56.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:09:56.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:09:56.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:09:56.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:09:56.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:09:56.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:09:56.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:09:56.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:09:56.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:09:56.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:09:56.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:09:56.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:09:56.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:09:56.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:09:56.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:09:56.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:09:56.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.333 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:56.333 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:56.333 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.846992813, 2021-01-15T19:09:56.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:09:56.334 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.334 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.335 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:09:56.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.335 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:56.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:09:56.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:09:56.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:09:56.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:09:56.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:09:56.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:09:56.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:09:56.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:09:56.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:09:56.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:09:56.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:09:56.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:09:56.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:09:56.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:09:56.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:09:56.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:09:56.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:56.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:09:56.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:09:56.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:09:56.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:09:56.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:09:56.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.348 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.349 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:56.349 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:56.349 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.845670947000002, 2021-01-15T19:09:56.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:09:56.350 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.350 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.351 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:09:56.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.351 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:09:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:09:56.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:09:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:09:56.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:09:56.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:09:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:09:56.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:09:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:09:56.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:09:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:09:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:09:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:09:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:09:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:09:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:09:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:09:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:09:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:09:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:09:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:09:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:09:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:56.365 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.845300694000002, 2021-01-15T19:09:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:09:56.366 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.367 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:09:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:09:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:09:56.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:09:56.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:09:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:09:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:09:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:09:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:09:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:09:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:09:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:09:56.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:09:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:09:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:09:56.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:09:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:09:56.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:09:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:09:56.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:09:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:09:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:09:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:09:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:09:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:56.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.844966300999999, 2021-01-15T19:09:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:09:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.385 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:09:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:56.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:09:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:09:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:09:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:09:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:09:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:09:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:09:56.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:09:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:09:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:09:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:09:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:09:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:09:56.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:09:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:09:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:09:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:09:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:09:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:09:56.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:09:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.404 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:09:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 62.903705041999999, 2021-01-15T19:09:56.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:09:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:09:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:09:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:09:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:09:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:09:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:09:56.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:09:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:09:56.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:09:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:09:56.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:09:56.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:09:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:09:56.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:09:56.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:09:56.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:09:56.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:09:56.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:09:56.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:09:56.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:09:56.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.419 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.420 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:09:56.421 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:56.421 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.911342169999998, 2021-01-15T19:09:56.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:09:56.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.422 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.423 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:09:56.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.423 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:56.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:09:56.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:09:56.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:09:56.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:09:56.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:09:56.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:09:56.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:09:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:09:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:09:56.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:09:56.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:09:56.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:09:56.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:09:56.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:09:56.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:09:56.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:09:56.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:09:56.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:09:56.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:09:56.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:09:56.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:09:56.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:09:56.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:09:56.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:09:56.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:09:56.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:09:56.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:09:56.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.438 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.438 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:09:56.439 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:56.439 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.881347966, 2021-01-15T19:09:56.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:09:56.439 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.439 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.440 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.440 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:09:56.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:56.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:09:56.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:09:56.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:09:56.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:09:56.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:09:56.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:09:56.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:09:56.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:09:56.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:09:56.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:09:56.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:09:56.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:09:56.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:09:56.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:09:56.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:09:56.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:09:56.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:09:56.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:09:56.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:09:56.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:09:56.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.452 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.453 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.453 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:56.454 INFO:tasks.rados.rados.0.smithi001.stdout:487: done (0 left) 2021-01-15T19:09:56.455 INFO:tasks.rados.rados.0.smithi001.stdout:488: snap_create 2021-01-15T19:09:56.455 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:09:56.456 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:56.453+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:56.556 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:56.555+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:56.620 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:56.620 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:09:56.620 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:56.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:09:56.657 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:56.655+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:56.754 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:56.754 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:09:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:09:56.755 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:09:56.755 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:56.757 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:56.756+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:56.764 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:09:56.859 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:56.858+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:56.890 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:09:56.891 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:09:56.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:09:56.891 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:09:56.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.891 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:09:56.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:56.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.702862922999998, 2021-01-15T19:09:56.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:09:56.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.893 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:56.893 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:09:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:09:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:09:56.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:09:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:09:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:09:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:09:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:09:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:09:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:09:56.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:09:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:09:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:09:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:09:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:09:56.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:09:56.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:09:56.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:09:56.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.905 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.906 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:09:56.906 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:56.907 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.621905472999998, 2021-01-15T19:09:56.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:09:56.907 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.907 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:56.907 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:56.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:09:56.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:09:56.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:09:56.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:09:56.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:09:56.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:09:56.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:09:56.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:09:56.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:09:56.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:09:56.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:09:56.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:09:56.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:09:56.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:09:56.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:09:56.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:09:56.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:09:56.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:09:56.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:09:56.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:56.921 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.617494199000006, 2021-01-15T19:09:56.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:09:56.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:56.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:56.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:09:56.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:09:56.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:09:56.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:09:56.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:09:56.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:09:56.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:09:56.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:09:56.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:09:56.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:09:56.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:09:56.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:09:56.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:09:56.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:09:56.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:09:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:09:56.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:09:56.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:09:56.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.936 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.936 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.937 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:09:56.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:56.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.655374746000007, 2021-01-15T19:09:56.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:09:56.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:56.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.941 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.941 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:09:56.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.942 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:56.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:09:56.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:09:56.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:09:56.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:09:56.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:09:56.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:09:56.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:09:56.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:09:56.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:09:56.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:09:56.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:09:56.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:09:56.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:09:56.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:09:56.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:56.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:09:56.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:09:56.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:56.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:09:56.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:09:56.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:56.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:09:56.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:09:56.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:09:56.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:09:56.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.968 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:09:56.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:56.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.655226728000002, 2021-01-15T19:09:56.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:09:56.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:56.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:56.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:56.973 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:56.973 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:09:56.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.974 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:56.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:56.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:56.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:56.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:56.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:09:56.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:09:56.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:56.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:09:56.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:09:56.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:56.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:09:56.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:09:56.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:56.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:09:56.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:09:56.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:56.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:09:56.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:09:56.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:09:56.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:09:56.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:56.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:09:56.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:09:56.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:09:56.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:09:56.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:56.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:09:56.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:09:56.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:56.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:09:56.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:09:56.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:56.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:09:56.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:09:56.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:56.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:09:56.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:09:56.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.996 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:56.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:56.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:56.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:56.997 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=201, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.534791210999998, 2021-01-15T19:09:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999, 2021-01-15T19:09:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:09:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6137999999999998e-05 2021-01-15T19:09:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352839+0000", 2021-01-15T19:09:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2480000000000002e-06 2021-01-15T19:09:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352848+0000", 2021-01-15T19:09:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4456e-05 2021-01-15T19:09:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352863+0000", 2021-01-15T19:09:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.755e-06 2021-01-15T19:09:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352869+0000", 2021-01-15T19:09:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01049957 2021-01-15T19:09:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.363368+0000", 2021-01-15T19:09:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001221172 2021-01-15T19:09:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.364589+0000", 2021-01-15T19:09:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999 2021-01-15T19:09:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=202, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:09:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.534675929000002, 2021-01-15T19:09:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293, 2021-01-15T19:09:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:09:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1519e-05 2021-01-15T19:09:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352950+0000", 2021-01-15T19:09:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3610000000000001e-06 2021-01-15T19:09:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352958+0000", 2021-01-15T19:09:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4105e-05 2021-01-15T19:09:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352972+0000", 2021-01-15T19:09:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3709999999999999e-06 2021-01-15T19:09:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352978+0000", 2021-01-15T19:09:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013549736999999999 2021-01-15T19:09:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.366527+0000", 2021-01-15T19:09:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025352 2021-01-15T19:09:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.369063+0000", 2021-01-15T19:09:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293 2021-01-15T19:09:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=257, reqid=client.4418.0:863, at_version=85'126, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.515232396999998, 2021-01-15T19:09:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972, 2021-01-15T19:09:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:09:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000526152 2021-01-15T19:09:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372878+0000", 2021-01-15T19:09:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8391e-05 2021-01-15T19:09:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372906+0000", 2021-01-15T19:09:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2021-01-15T19:09:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372924+0000", 2021-01-15T19:09:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4420000000000001e-06 2021-01-15T19:09:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372931+0000", 2021-01-15T19:09:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040511399999999998 2021-01-15T19:09:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373336+0000", 2021-01-15T19:09:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4637000000000001e-05 2021-01-15T19:09:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373351+0000", 2021-01-15T19:09:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014127954 2021-01-15T19:09:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387479+0000", 2021-01-15T19:09:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010297 2021-01-15T19:09:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387582+0000", 2021-01-15T19:09:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972 2021-01-15T19:09:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=261, reqid=client.4418.0:849, at_version=85'130, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:09:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.488077556999997, 2021-01-15T19:09:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001, 2021-01-15T19:09:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:09:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041353340000000001 2021-01-15T19:09:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403642+0000", 2021-01-15T19:09:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6962999999999998e-05 2021-01-15T19:09:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403679+0000", 2021-01-15T19:09:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7178999999999999e-05 2021-01-15T19:09:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403696+0000", 2021-01-15T19:09:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:09:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403703+0000", 2021-01-15T19:09:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6183999999999997e-05 2021-01-15T19:09:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403749+0000", 2021-01-15T19:09:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5073e-05 2021-01-15T19:09:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403764+0000", 2021-01-15T19:09:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012428852000000001 2021-01-15T19:09:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416193+0000", 2021-01-15T19:09:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1882000000000002e-05 2021-01-15T19:09:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416245+0000", 2021-01-15T19:09:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001 2021-01-15T19:09:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.700905736999999, 2021-01-15T19:09:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:09:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:09:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:09:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:09:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:09:57.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:09:57.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:09:57.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:09:57.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:09:57.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:09:57.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:09:57.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:09:57.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:09:57.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:09:57.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:09:57.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:09:57.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:09:57.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:09:57.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:09:57.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:09:57.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:09:57.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:09:57.061 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.061 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.061 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.062 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:57.062 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:57.062 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.700714447999999, 2021-01-15T19:09:57.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:09:57.062 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.063 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.063 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:09:57.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.064 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:57.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:09:57.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:09:57.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:09:57.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:09:57.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:09:57.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:09:57.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:09:57.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:09:57.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:09:57.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:09:57.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:09:57.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:09:57.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:09:57.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:09:57.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:09:57.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:09:57.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:09:57.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:09:57.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:09:57.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:09:57.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.075 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.076 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:09:57.076 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:57.076 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.700552094999999, 2021-01-15T19:09:57.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:09:57.077 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.077 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.077 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.078 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.078 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:09:57.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.078 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:57.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:09:57.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:09:57.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:09:57.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:09:57.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:09:57.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:09:57.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:09:57.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:09:57.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:09:57.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:09:57.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:09:57.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:09:57.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:09:57.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:09:57.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:09:57.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:09:57.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:09:57.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:09:57.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:09:57.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:09:57.089 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.089 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.090 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:942 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~2203648,checksum] snapc 0=[] ondisk+read+known_if_redirected e90)", 2021-01-15T19:09:57.090 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:57.091 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.431821243000002, 2021-01-15T19:09:57.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817, 2021-01-15T19:09:57.091 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.091 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.091 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.092 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 942 2021-01-15T19:09:57.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.092 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:57.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:09:57.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3491000000000006e-05 2021-01-15T19:09:57.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455826+0000", 2021-01-15T19:09:57.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1099999999999997e-06 2021-01-15T19:09:57.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455834+0000", 2021-01-15T19:09:57.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2245e-05 2021-01-15T19:09:57.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455846+0000", 2021-01-15T19:09:57.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1660000000000002e-06 2021-01-15T19:09:57.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455851+0000", 2021-01-15T19:09:57.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001470865 2021-01-15T19:09:57.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457322+0000", 2021-01-15T19:09:57.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012032 2021-01-15T19:09:57.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457442+0000", 2021-01-15T19:09:57.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097176199999999997 2021-01-15T19:09:57.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.467160+0000", 2021-01-15T19:09:57.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817 2021-01-15T19:09:57.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.102 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.103 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s1 91/90 ECSubReadReply(tid=71, attrs_read=0))", 2021-01-15T19:09:57.103 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:57.103 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.671296847999997, 2021-01-15T19:09:57.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001, 2021-01-15T19:09:57.104 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.104 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.104 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:57.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:09:57.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032454459999999999 2021-01-15T19:09:57.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219533+0000", 2021-01-15T19:09:57.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2258e-05 2021-01-15T19:09:57.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219545+0000", 2021-01-15T19:09:57.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3614999999999999e-05 2021-01-15T19:09:57.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219559+0000", 2021-01-15T19:09:57.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2020000000000001e-06 2021-01-15T19:09:57.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219565+0000", 2021-01-15T19:09:57.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4172999999999998e-05 2021-01-15T19:09:57.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219619+0000", 2021-01-15T19:09:57.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098911899999999994 2021-01-15T19:09:57.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229510+0000", 2021-01-15T19:09:57.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001 2021-01-15T19:09:57.112 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.112 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.113 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 96/92 ECSubWrite(tid=404, reqid=osd.4.0:404, at_version=96'68, trim_to=0'0, roll_forward_to=93'66))", 2021-01-15T19:09:57.113 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:57.114 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.675444331000001, 2021-01-15T19:09:57.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125, 2021-01-15T19:09:57.114 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.114 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.115 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:57.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:09:57.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1780000000000002e-05 2021-01-15T19:09:57.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212192+0000", 2021-01-15T19:09:57.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2773e-05 2021-01-15T19:09:57.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212214+0000", 2021-01-15T19:09:57.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8345000000000001e-05 2021-01-15T19:09:57.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212233+0000", 2021-01-15T19:09:57.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8379999999999996e-06 2021-01-15T19:09:57.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212240+0000", 2021-01-15T19:09:57.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1579000000000002e-05 2021-01-15T19:09:57.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212291+0000", 2021-01-15T19:09:57.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3115999999999999e-05 2021-01-15T19:09:57.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212304+0000", 2021-01-15T19:09:57.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011237044 2021-01-15T19:09:57.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223541+0000", 2021-01-15T19:09:57.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9774999999999999e-05 2021-01-15T19:09:57.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223571+0000", 2021-01-15T19:09:57.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125 2021-01-15T19:09:57.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.126 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:982 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:57.126 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:57.126 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.668892945000003, 2021-01-15T19:09:57.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641, 2021-01-15T19:09:57.127 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.127 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.128 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 982 2021-01-15T19:09:57.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:57.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:09:57.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8164999999999999e-05 2021-01-15T19:09:57.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218719+0000", 2021-01-15T19:09:57.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6679999999999999e-06 2021-01-15T19:09:57.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218723+0000", 2021-01-15T19:09:57.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0639999999999999e-06 2021-01-15T19:09:57.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218729+0000", 2021-01-15T19:09:57.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2021-01-15T19:09:57.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218732+0000", 2021-01-15T19:09:57.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033278300000000002 2021-01-15T19:09:57.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219064+0000", 2021-01-15T19:09:57.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118377 2021-01-15T19:09:57.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219183+0000", 2021-01-15T19:09:57.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014915026 2021-01-15T19:09:57.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.234098+0000", 2021-01-15T19:09:57.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641 2021-01-15T19:09:57.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.139 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:57.139 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:57.139 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.668832182000003, 2021-01-15T19:09:57.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:09:57.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.140 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.140 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:09:57.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.141 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:57.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:09:57.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:09:57.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:09:57.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:09:57.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:09:57.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:09:57.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:09:57.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:09:57.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:09:57.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:09:57.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:09:57.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:09:57.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:09:57.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:09:57.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:09:57.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:09:57.150 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.151 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:09:57.152 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:57.152 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.668770492999997, 2021-01-15T19:09:57.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:09:57.152 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.152 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:09:57.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.153 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.153 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:09:57.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.154 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:57.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:09:57.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:09:57.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:09:57.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:09:57.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:09:57.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:09:57.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:09:57.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:09:57.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:09:57.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:09:57.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:09:57.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:09:57.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:09:57.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:09:57.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:09:57.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:09:57.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.163 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.163 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.164 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 110/52 ECSubWrite(tid=513, reqid=client.4418.0:1096, at_version=110'199, trim_to=0'0, roll_forward_to=109'197))", 2021-01-15T19:09:57.164 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:57.164 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.670109946, 2021-01-15T19:09:57.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999, 2021-01-15T19:09:57.165 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.165 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.165 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:57.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:09:57.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091685599999999996 2021-01-15T19:09:57.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218391+0000", 2021-01-15T19:09:57.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6659999999999998e-05 2021-01-15T19:09:57.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218428+0000", 2021-01-15T19:09:57.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6130999999999999e-05 2021-01-15T19:09:57.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218444+0000", 2021-01-15T19:09:57.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-05 2021-01-15T19:09:57.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218455+0000", 2021-01-15T19:09:57.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0050000000000003e-05 2021-01-15T19:09:57.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218515+0000", 2021-01-15T19:09:57.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4779e-05 2021-01-15T19:09:57.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218530+0000", 2021-01-15T19:09:57.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014431424 2021-01-15T19:09:57.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.232961+0000", 2021-01-15T19:09:57.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.893e-05 2021-01-15T19:09:57.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233020+0000", 2021-01-15T19:09:57.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999 2021-01-15T19:09:57.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.176 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1142 3.9s0 3:935d2ce1:::smithi00140323-50:head [create,setxattr nOg\\b]PYmQpZprhrNSoOiXch_OVhaejNkYNj\\rrb (1507) in=40b,setxattr WcdVgSb^Xpei^gpqgVORjX_ifYYlebi^lg_\\ (1997) in=36b,setxattr aQpVtT`^]OieemiqNUnaUfic[c]Wqmdis`XQ_Tqm (908) in=40b,setxattr nYb]ZVbStbOagQf^W[sgda\\nc]R]dSaefWh]adhg (1893) in=40b,setxattr U^ja\\c]ahh_WgcSQkePloVfUV[_dZZa_Qa^jZsgO (1086) in=40b,setxattr TUgWsbVkpRdhTUXqjX]eOikS`Wa]sVkaNpWTYbY[ (1399) in=40b,setxattr gXqmVqsjhpeU\\nn\\SoQ`SZfYeg^]omdk_pVTP`do (806) in=40b,setxattr TrVPgo]^oWNoalZnotgZbhqliYh[id^Q_loQnh^Z (1963) in=40b,setxattr _TS_[m^pOge]N^c\\[mgPssoXoqlONURs`Shj\\UYk (1647) in=40b,setxattr W^YiocPQgkgoftgn`q_YOW_UUYQpSd_s[^sOQfqh (874) in=40b,setxattr lbeYtWUYqpORgbP`aWmV\\TSo[odhaTnocqnTrlQr (945) in=40b,setxattr QdO^gYZc\\rqThdOUbdYlmofVstjYkpQs_ab]o_jT (1786) in=40b,setxattr hNj]sQWrnXhXNh[PPa`V^jVe_RrNYUN^^^\\VmZj[ (943) in=40b,setxattr s_fmbQYrifmdiN_RPaXpdaZd[\\WffOb^[VNTtQjW (383) in=40b,setxattr _kZU^ZanaPRW_XWjbYlgtXRWi^T[O^TVpkTs`npQ (1609) in=40b,setxattr qaNdNbjbVYZU[Ygrejofh`kRTiOS_jZ_PkYZ^U[p (733) in=40b,setxattr i_NbbQjhTiYWVccoc`[glilPmsbWr]RalgkhRmcc (417) in=40b,setxattr aZess_ORomNedc_mbO_QcQqabPbRt_aa\\dslZoPr (17332) in=40b,setxattr Roai^sie^qdaa[nObntZWmUWiV]flrp`qaaVRecS (1785) in=40b,setxattr ek^QaVgeZSSY_ck^tsPa^ncnge]eT_asrQ_NOa\\Q (1153) in=40b] snapc 5c=[5c,58,57,56,53,4b,44,41,26] ondisk+write+known_if_redirected e119)", 2021-01-15T19:09:57.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:57.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.6082585619999996, 2021-01-15T19:09:57.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875, 2021-01-15T19:09:57.177 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:09:57.177 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:09:57.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:09:57.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.178 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1142 2021-01-15T19:09:57.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.179 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:09:57.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:09:57.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:57.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:09:57.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:09:57.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:09:57.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4063999999999999e-05 2021-01-15T19:09:57.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:09:57.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279420+0000", 2021-01-15T19:09:57.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2021-01-15T19:09:57.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:09:57.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279428+0000", 2021-01-15T19:09:57.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3764999999999999e-05 2021-01-15T19:09:57.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:09:57.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279441+0000", 2021-01-15T19:09:57.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4460000000000004e-06 2021-01-15T19:09:57.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279449+0000", 2021-01-15T19:09:57.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5884000000000003e-05 2021-01-15T19:09:57.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:09:57.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279495+0000", 2021-01-15T19:09:57.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014788 2021-01-15T19:09:57.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:09:57.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279643+0000", 2021-01-15T19:09:57.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104872 2021-01-15T19:09:57.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.280748+0000", 2021-01-15T19:09:57.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044365569999999998 2021-01-15T19:09:57.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.285184+0000", 2021-01-15T19:09:57.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064052199999999997 2021-01-15T19:09:57.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:09:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291589+0000", 2021-01-15T19:09:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4213000000000002e-05 2021-01-15T19:09:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:09:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:09:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:09:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291664+0000", 2021-01-15T19:09:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875 2021-01-15T19:09:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:09:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:09:57.192 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:09:57.194 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:57.192+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:57.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:09:57.222 INFO:tasks.rados.rados.0.smithi001.stdout:488: done (0 left) 2021-01-15T19:09:57.223 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 17 current snap is 56 2021-01-15T19:09:57.223 INFO:tasks.rados.rados.0.smithi001.stdout:489: seq_num 175 ranges {0=540672} 2021-01-15T19:09:57.226 INFO:tasks.rados.rados.0.smithi001.stdout:489: writing smithi00140323-17 from 0 to 540672 tid 1 2021-01-15T19:09:57.226 INFO:tasks.rados.rados.0.smithi001.stdout:490: rollback oid 49 current snap is 56 2021-01-15T19:09:57.226 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 49 to 46 2021-01-15T19:09:57.227 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 37 current snap is 56 2021-01-15T19:09:57.227 INFO:tasks.rados.rados.0.smithi001.stdout:491: seq_num 176 ranges {4505600=557056} 2021-01-15T19:09:57.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:09:57.256 INFO:tasks.rados.rados.0.smithi001.stdout:491: writing smithi00140323-37 from 4505600 to 5062656 tid 1 2021-01-15T19:09:57.256 INFO:tasks.rados.rados.0.smithi001.stdout:492: read oid 9 snap 38 2021-01-15T19:09:57.256 INFO:tasks.rados.rados.0.smithi001.stdout:492: expect (ObjNum 139 snap 34 seq_num 139) 2021-01-15T19:09:57.257 INFO:tasks.rados.rados.0.smithi001.stdout:493: rollback oid 16 current snap is 56 2021-01-15T19:09:57.257 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 16 to 51 2021-01-15T19:09:57.257 INFO:tasks.rados.rados.0.smithi001.stdout:489: finishing write tid 1 to smithi00140323-17 2021-01-15T19:09:57.257 INFO:tasks.rados.rados.0.smithi001.stdout:490: finishing rollback tid 0 to smithi00140323-49 2021-01-15T19:09:57.257 INFO:tasks.rados.rados.0.smithi001.stdout:490: finishing rollback tid 1 to smithi00140323-49 2021-01-15T19:09:57.258 INFO:tasks.rados.rados.0.smithi001.stdout:490: finishing rollback tid 2 to smithi00140323-49 2021-01-15T19:09:57.258 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 102 (ObjNum 122 snap 27 seq_num 122) dirty exists 2021-01-15T19:09:57.258 INFO:tasks.rados.rados.0.smithi001.stdout:489: finishing write tid 2 to smithi00140323-17 2021-01-15T19:09:57.258 INFO:tasks.rados.rados.0.smithi001.stdout:489: finishing write tid 3 to smithi00140323-17 2021-01-15T19:09:57.258 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 17 v 92 (ObjNum 175 snap 56 seq_num 175) dirty exists 2021-01-15T19:09:57.259 INFO:tasks.rados.rados.0.smithi001.stdout:489: left oid 17 (ObjNum 175 snap 56 seq_num 175) 2021-01-15T19:09:57.259 INFO:tasks.rados.rados.0.smithi001.stdout:489: done (4 left) 2021-01-15T19:09:57.259 INFO:tasks.rados.rados.0.smithi001.stdout:490: done (3 left) 2021-01-15T19:09:57.259 INFO:tasks.rados.rados.0.smithi001.stdout:494: snap_remove snap 53 2021-01-15T19:09:57.295 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:57.293+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:57.356 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:57.357 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:57.357 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:57.358 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:57.368 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:09:57.408 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:57.407+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:57.497 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:57.496+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:57.539 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:57.539 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:57.540 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:09:57.540 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:09:57.540 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:57.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:09:57.633 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:57.632+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:57.699 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:57.698+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:57.723 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:57.723 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:09:57.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:09:57.723 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:09:57.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.724 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:434 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~663552] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:09:57.724 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:09:57.724 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 78.461937306999999, 2021-01-15T19:09:57.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022929644999999999, 2021-01-15T19:09:57.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 434 2021-01-15T19:09:57.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.726 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:09:57.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:09:57.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044819800000000002 2021-01-15T19:09:57.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258996+0000", 2021-01-15T19:09:57.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4090000000000001e-05 2021-01-15T19:09:57.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259011+0000", 2021-01-15T19:09:57.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7579000000000001e-05 2021-01-15T19:09:57.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259038+0000", 2021-01-15T19:09:57.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1133e-05 2021-01-15T19:09:57.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259049+0000", 2021-01-15T19:09:57.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3312000000000006e-05 2021-01-15T19:09:57.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259123+0000", 2021-01-15T19:09:57.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3225e-05 2021-01-15T19:09:57.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:09:57.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259136+0000", 2021-01-15T19:09:57.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002543362 2021-01-15T19:09:57.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261679+0000", 2021-01-15T19:09:57.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012904299999999999 2021-01-15T19:09:57.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261808+0000", 2021-01-15T19:09:57.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001322642 2021-01-15T19:09:57.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.263131+0000", 2021-01-15T19:09:57.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012144998000000001 2021-01-15T19:09:57.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275276+0000", 2021-01-15T19:09:57.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060959580000000003 2021-01-15T19:09:57.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.281372+0000", 2021-01-15T19:09:57.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106105 2021-01-15T19:09:57.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.281478+0000", 2021-01-15T19:09:57.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022929644999999999 2021-01-15T19:09:57.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.741 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.742 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:09:57.743 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:09:57.743 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.467223794999995, 2021-01-15T19:09:57.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:09:57.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.744 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.744 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.744 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:09:57.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.745 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:09:57.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:09:57.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:09:57.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:09:57.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:09:57.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:09:57.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:09:57.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:09:57.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:09:57.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:09:57.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:09:57.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:09:57.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:09:57.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:09:57.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:09:57.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:09:57.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:09:57.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:09:57.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:09:57.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:09:57.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:09:57.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:09:57.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:09:57.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.758 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:09:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:09:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.467058682000001, 2021-01-15T19:09:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:09:57.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.760 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.761 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:09:57.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.761 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:09:57.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:09:57.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:09:57.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:09:57.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:09:57.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:09:57.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:09:57.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:09:57.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:09:57.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:09:57.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:09:57.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:09:57.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:09:57.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:57.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:09:57.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:09:57.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:09:57.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:09:57.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:57.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:09:57.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:09:57.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:09:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:09:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:09:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:09:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:09:57.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:09:57.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:09:57.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:09:57.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.459778632999999, 2021-01-15T19:09:57.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:09:57.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.778 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:09:57.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:09:57.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:09:57.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:09:57.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:09:57.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:09:57.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:09:57.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:09:57.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:09:57.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:09:57.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:09:57.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:09:57.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:09:57.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:09:57.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:09:57.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:09:57.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:09:57.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:09:57.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:09:57.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:09:57.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:09:57.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:09:57.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:09:57.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:09:57.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.793 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:09:57.793 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:09:57.793 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.459651999999998, 2021-01-15T19:09:57.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:09:57.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:57.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:09:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:09:57.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:09:57.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:09:57.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:09:57.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:09:57.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:09:57.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:09:57.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:09:57.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:09:57.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:09:57.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:09:57.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:09:57.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:09:57.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:57.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:09:57.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:09:57.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:09:57.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:09:57.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:57.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:09:57.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:09:57.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:09:57.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:09:57.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:09:57.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:09:57.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:09:57.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:09:57.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.809 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.810 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:09:57.810 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:09:57.810 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.376915035000003, 2021-01-15T19:09:57.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:09:57.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.811 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:09:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:09:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:09:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:09:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:09:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:09:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:09:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:09:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:09:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:09:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:09:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:09:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:09:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:09:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:09:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:09:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:09:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:09:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:09:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:09:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:09:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:09:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:09:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:09:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:09:57.825 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:09:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.376765421000002, 2021-01-15T19:09:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:09:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:09:57.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:09:57.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:09:57.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:09:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:09:57.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:09:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:09:57.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:09:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:09:57.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:09:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:09:57.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:09:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:09:57.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:09:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:09:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:09:57.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:09:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:09:57.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:09:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:57.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:09:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:09:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:09:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:09:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:09:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:09:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:09:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:09:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:09:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.356467739000003, 2021-01-15T19:09:57.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:09:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.843 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:09:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:09:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:09:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:09:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:09:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:09:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:09:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:09:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:09:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:09:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:09:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:09:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:09:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:09:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:09:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:09:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:09:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:09:57.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:09:57.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:09:57.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:09:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:09:57.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:09:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:09:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:09:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:09:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:09:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:09:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.353411088000001, 2021-01-15T19:09:57.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:09:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:09:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:09:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:09:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:09:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:09:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:09:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:09:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:09:57.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:09:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:09:57.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:09:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:09:57.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:09:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:09:57.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:09:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:09:57.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:09:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:09:57.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:09:57.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:09:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:09:57.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:09:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:09:57.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:09:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:09:57.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:09:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:09:57.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:09:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:09:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.350397825999998, 2021-01-15T19:09:57.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:09:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:09:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:09:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:09:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:09:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:09:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:09:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:09:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:09:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:09:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:09:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:09:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:09:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:09:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:09:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:09:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:09:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:09:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:09:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:09:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:09:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:09:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:09:57.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:09:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:09:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:09:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:09:57.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:09:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:09:57.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:09:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.347650215000002, 2021-01-15T19:09:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:09:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:09:57.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:09:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:09:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:09:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:09:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:09:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:09:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:09:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:09:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:09:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:09:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:09:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:09:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:09:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:09:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:09:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:09:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:09:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:09:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:09:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:09:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:09:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:09:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:09:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:09:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:09:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:09:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:09:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.912 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:57.913 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:09:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.347216392999997, 2021-01-15T19:09:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:09:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.914 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:09:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.915 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:09:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:09:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:09:57.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:09:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:09:57.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:09:57.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:09:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:09:57.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:09:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:09:57.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:09:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:09:57.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:09:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:09:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:09:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:09:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:09:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:09:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:09:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:09:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:09:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:09:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:09:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:09:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:09:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:09:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:09:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:09:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:09:57.931 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.347092234000002, 2021-01-15T19:09:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:09:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:57.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:09:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.933 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:09:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:09:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:09:57.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:09:57.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:09:57.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:09:57.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:09:57.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:09:57.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:09:57.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:09:57.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:09:57.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:09:57.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:09:57.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:09:57.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:09:57.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:09:57.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:09:57.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:57.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:09:57.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:09:57.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:09:57.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:09:57.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:09:57.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:09:57.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:09:57.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:09:57.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.947 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.948 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:09:57.948 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:09:57.948 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.242415698999999, 2021-01-15T19:09:57.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:09:57.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:09:57.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:57.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:57.949 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:57.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:09:57.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.950 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:09:57.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:09:57.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:09:57.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:09:57.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:09:57.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:09:57.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:09:57.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:09:57.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:09:57.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:09:57.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:09:57.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:09:57.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:09:57.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:09:57.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:09:57.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:09:57.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:09:57.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:09:57.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:09:57.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:09:57.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:09:57.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:09:57.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:09:57.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:57.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:09:57.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:09:57.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:09:57.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:09:57.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:09:57.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:09:57.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:09:57.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:09:57.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:09:57.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:09:57.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:09:57.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:09:57.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:09:57.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:09:57.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:09:57.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:09:57.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:09:57.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.969 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:09:57.970 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:09:57.970 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.362194619999997, 2021-01-15T19:09:57.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:09:57.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:57.971 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:09:57.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:09:57.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:09:57.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:09:57.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:09:57.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:09:57.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:09:57.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:09:57.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:09:57.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:09:57.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:09:57.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:09:57.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:09:57.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:57.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:09:57.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:09:57.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:09:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:09:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.980 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:09:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:09:57.981 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.332476462999999, 2021-01-15T19:09:57.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:09:57.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.982 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:09:57.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:09:57.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:09:57.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:09:57.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:09:57.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:09:57.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:09:57.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:09:57.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:09:57.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:09:57.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:09:57.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:57.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:09:57.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:09:57.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:57.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:09:57.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:09:57.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.991 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:57.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:57.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.992 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:09:57.992 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:09:57.992 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.328390296000002, 2021-01-15T19:09:57.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:09:57.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:57.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:57.993 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:57.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:57.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:09:57.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:57.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:57.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:09:57.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:09:57.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:57.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:09:57.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:09:57.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:57.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:09:57.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:09:57.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:57.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:09:57.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:09:57.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:57.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:09:57.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:09:57.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:57.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:57.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:58.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:09:58.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:09:58.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:58.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:09:58.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:09:58.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:58.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:09:58.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:09:58.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.328254545, 2021-01-15T19:09:58.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:09:58.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:58.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:58.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:58.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:58.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:09:58.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:58.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:58.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:09:58.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:09:58.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:58.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:09:58.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:09:58.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:58.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:09:58.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:09:58.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:58.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:09:58.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:09:58.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:58.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:09:58.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:09:58.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:58.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:09:58.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:09:58.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:58.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:09:58.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:09:58.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:58.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=28'16, committed=1, applied=1))", 2021-01-15T19:09:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:09:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.328027695999999, 2021-01-15T19:09:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022959151000000001, 2021-01-15T19:09:58.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:58.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:09:58.014 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:58.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:58.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:09:58.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:58.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:58.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:09:58.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6398999999999997e-05 2021-01-15T19:09:58.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:58.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392544+0000", 2021-01-15T19:09:58.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1029999999999999e-05 2021-01-15T19:09:58.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:58.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392555+0000", 2021-01-15T19:09:58.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4134999999999999e-05 2021-01-15T19:09:58.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:58.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392569+0000", 2021-01-15T19:09:58.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.874e-06 2021-01-15T19:09:58.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:58.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392575+0000", 2021-01-15T19:09:58.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022133907000000001 2021-01-15T19:09:58.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:58.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414709+0000", 2021-01-15T19:09:58.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070780599999999995 2021-01-15T19:09:58.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:58.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415417+0000", 2021-01-15T19:09:58.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022959151000000001 2021-01-15T19:09:58.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.022 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:58.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:09:58.023 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:09:58.023 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.489755562, 2021-01-15T19:09:58.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:09:58.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:09:58.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:09:58.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:09:58.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:09:58.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:09:58.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:09:58.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:09:58.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:09:58.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:09:58.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:09:58.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:09:58.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:09:58.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:09:58.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:09:58.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:09:58.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:09:58.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:09:58.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:09:58.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:09:58.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:09:58.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:09:58.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:09:58.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:09:58.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:09:58.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:09:58.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:58.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:09:58.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:09:58.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:09:58.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:09:58.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:09:58.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:09:58.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:09:58.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:09:58.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:09:58.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:09:58.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:09:58.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:09:58.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:09:58.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:09:58.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:09:58.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:09:58.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:58.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:09:58.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:09:58.038 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:58.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:09:58.039 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2021-01-15T19:09:58.039 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:09:58.039 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2021-01-15T19:09:58.040 INFO:tasks.ceph.osd.6:Restarting daemon 2021-01-15T19:09:58.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-01-15T19:09:58.042 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:58.038+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:58.042 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:09:58.044 INFO:tasks.ceph.osd.6:Started 2021-01-15T19:09:58.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:09:58.141 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:58.140+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:58.177 INFO:teuthology.orchestra.run.smithi136.stderr:no valid command found; 10 closest matches: 2021-01-15T19:09:58.177 INFO:teuthology.orchestra.run.smithi136.stderr:0 2021-01-15T19:09:58.178 INFO:teuthology.orchestra.run.smithi136.stderr:1 2021-01-15T19:09:58.178 INFO:teuthology.orchestra.run.smithi136.stderr:2 2021-01-15T19:09:58.178 INFO:teuthology.orchestra.run.smithi136.stderr:abort 2021-01-15T19:09:58.178 INFO:teuthology.orchestra.run.smithi136.stderr:assert 2021-01-15T19:09:58.179 INFO:teuthology.orchestra.run.smithi136.stderr:config diff 2021-01-15T19:09:58.179 INFO:teuthology.orchestra.run.smithi136.stderr:config diff get 2021-01-15T19:09:58.179 INFO:teuthology.orchestra.run.smithi136.stderr:config get 2021-01-15T19:09:58.179 INFO:teuthology.orchestra.run.smithi136.stderr:config help [] 2021-01-15T19:09:58.179 INFO:teuthology.orchestra.run.smithi136.stderr:config set ... 2021-01-15T19:09:58.180 INFO:teuthology.orchestra.run.smithi136.stderr:admin_socket: invalid command 2021-01-15T19:09:58.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 2021-01-15T19:09:58.181 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2021-01-15T19:09:58.196 INFO:tasks.rados.rados.0.smithi001.stdout:494: done (3 left) 2021-01-15T19:09:58.196 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 29 current snap is 56 2021-01-15T19:09:58.197 INFO:tasks.rados.rados.0.smithi001.stdout:495: seq_num 177 ranges {2056192=573440} 2021-01-15T19:09:58.204 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:09:58.212 INFO:tasks.rados.rados.0.smithi001.stdout:495: writing smithi00140323-29 from 2056192 to 2629632 tid 1 2021-01-15T19:09:58.213 INFO:tasks.rados.rados.0.smithi001.stdout:496: rmattr oid 25 current snap is 56 2021-01-15T19:09:58.213 INFO:tasks.rados.rados.0.smithi001.stdout:496: done (4 left) 2021-01-15T19:09:58.213 INFO:tasks.rados.rados.0.smithi001.stdout:497: delete oid 4 current snap is 56 2021-01-15T19:09:58.213 INFO:tasks.rados.rados.0.smithi001.stdout:493: finishing rollback tid 1 to smithi00140323-16 2021-01-15T19:09:58.214 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 126 (ObjNum 933193800 snap 32581 seq_num 1) dirty dne 2021-01-15T19:09:58.214 INFO:tasks.rados.rados.0.smithi001.stdout:491: finishing write tid 1 to smithi00140323-37 2021-01-15T19:09:58.214 INFO:tasks.rados.rados.0.smithi001.stdout:491: finishing write tid 2 to smithi00140323-37 2021-01-15T19:09:58.214 INFO:tasks.rados.rados.0.smithi001.stdout:492: expect (ObjNum 139 snap 34 seq_num 139) 2021-01-15T19:09:58.240 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:09:58.241 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:09:58.241 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:09:58.241 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:09:58.243 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:58.242+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:58.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:09:58.261 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:09:58.282 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:09:58.298 INFO:tasks.rados.rados.0.smithi001.stdout:491: finishing write tid 3 to smithi00140323-37 2021-01-15T19:09:58.299 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 95 (ObjNum 176 snap 56 seq_num 176) dirty exists 2021-01-15T19:09:58.299 INFO:tasks.rados.rados.0.smithi001.stdout:491: left oid 37 (ObjNum 176 snap 56 seq_num 176) 2021-01-15T19:09:58.299 INFO:tasks.rados.rados.0.smithi001.stdout:491: done (4 left) 2021-01-15T19:09:58.299 INFO:tasks.rados.rados.0.smithi001.stdout:492: done (3 left) 2021-01-15T19:09:58.300 INFO:tasks.rados.rados.0.smithi001.stdout:493: done (2 left) 2021-01-15T19:09:58.300 INFO:tasks.rados.rados.0.smithi001.stdout:497: done (1 left) 2021-01-15T19:09:58.300 INFO:tasks.rados.rados.0.smithi001.stdout:498: read oid 20 snap -1 2021-01-15T19:09:58.300 INFO:tasks.rados.rados.0.smithi001.stdout:498: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:09:58.300 INFO:tasks.rados.rados.0.smithi001.stdout:495: finishing write tid 1 to smithi00140323-29 2021-01-15T19:09:58.301 INFO:tasks.rados.rados.0.smithi001.stdout:495: finishing write tid 2 to smithi00140323-29 2021-01-15T19:09:58.301 INFO:tasks.rados.rados.0.smithi001.stdout:495: finishing write tid 3 to smithi00140323-29 2021-01-15T19:09:58.301 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 137 (ObjNum 177 snap 56 seq_num 177) dirty exists 2021-01-15T19:09:58.301 INFO:tasks.rados.rados.0.smithi001.stdout:495: left oid 29 (ObjNum 177 snap 56 seq_num 177) 2021-01-15T19:09:58.302 INFO:tasks.rados.rados.0.smithi001.stdout:495: done (1 left) 2021-01-15T19:09:58.302 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 25 current snap is 56 2021-01-15T19:09:58.302 INFO:tasks.rados.rados.0.smithi001.stdout:499: seq_num 179 ranges {409600=606208} 2021-01-15T19:09:58.304 INFO:tasks.rados.rados.0.smithi001.stdout:499: writing smithi00140323-25 from 409600 to 1015808 tid 1 2021-01-15T19:09:58.305 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 36 current snap is 56 2021-01-15T19:09:58.305 INFO:tasks.rados.rados.0.smithi001.stdout:498: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:09:58.306 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:09:58.314 INFO:tasks.rados.rados.0.smithi001.stdout:500: seq_num 180 ranges {2703360=622592} 2021-01-15T19:09:58.333 INFO:tasks.rados.rados.0.smithi001.stdout:500: writing smithi00140323-36 from 2703360 to 3325952 tid 1 2021-01-15T19:09:58.333 INFO:tasks.rados.rados.0.smithi001.stdout:498: done (2 left) 2021-01-15T19:09:58.333 INFO:tasks.rados.rados.0.smithi001.stdout:501: snap_create 2021-01-15T19:09:58.333 INFO:tasks.rados.rados.0.smithi001.stdout:499: finishing write tid 1 to smithi00140323-25 2021-01-15T19:09:58.333 INFO:tasks.rados.rados.0.smithi001.stdout:499: finishing write tid 2 to smithi00140323-25 2021-01-15T19:09:58.334 INFO:tasks.rados.rados.0.smithi001.stdout:499: finishing write tid 3 to smithi00140323-25 2021-01-15T19:09:58.334 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 220 (ObjNum 179 snap 56 seq_num 179) dirty exists 2021-01-15T19:09:58.334 INFO:tasks.rados.rados.0.smithi001.stdout:499: left oid 25 (ObjNum 179 snap 56 seq_num 179) 2021-01-15T19:09:58.340 INFO:tasks.rados.rados.0.smithi001.stdout:500: finishing write tid 1 to smithi00140323-36 2021-01-15T19:09:58.341 INFO:tasks.rados.rados.0.smithi001.stdout:500: finishing write tid 2 to smithi00140323-36 2021-01-15T19:09:58.342 INFO:tasks.rados.rados.0.smithi001.stdout:500: finishing write tid 3 to smithi00140323-36 2021-01-15T19:09:58.343 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 133 (ObjNum 180 snap 56 seq_num 180) dirty exists 2021-01-15T19:09:58.343 INFO:tasks.rados.rados.0.smithi001.stdout:500: left oid 36 (ObjNum 180 snap 56 seq_num 180) 2021-01-15T19:09:58.346 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:58.344+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:58.346 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:09:58.367 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:09:58.389 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:09:58.410 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:09:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.411 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.6 2021-01-15T19:09:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: problem getting command descriptions from osd.6 2021-01-15T19:09:58.412 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:09:58.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:09:58.413 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:09:58.414 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:58.414 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:58.424 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:09:58.466 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:58.464+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:58.548 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:58.547+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:58.597 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:58.598 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:58.610 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:09:58.650 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:09:58.649+0000 7f3fb515a080 -1 Falling back to public interface 2021-01-15T19:09:58.654 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:09:58.653+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:09:58.743 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:58.743 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:58.749 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:58.748+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:09:58.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:09:58.851 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:58.850+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:58.924 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:58.924 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:58.934 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:09:58.978 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:58.977+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:59.053 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:59.052+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:59.065 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:59.066 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:59.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:09:59.159 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:59.158+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:59.249 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:59.249 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:59.255 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:59.254+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:59.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:09:59.313 INFO:tasks.rados.rados.0.smithi001.stdout:499: done (2 left) 2021-01-15T19:09:59.314 INFO:tasks.rados.rados.0.smithi001.stdout:500: done (1 left) 2021-01-15T19:09:59.314 INFO:tasks.rados.rados.0.smithi001.stdout:501: done (0 left) 2021-01-15T19:09:59.314 INFO:tasks.rados.rados.0.smithi001.stdout:502: rollback oid 1 current snap is 57 2021-01-15T19:09:59.314 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 1 to 38 2021-01-15T19:09:59.315 INFO:tasks.rados.rados.0.smithi001.stdout:503: snap_create 2021-01-15T19:09:59.319 INFO:tasks.rados.rados.0.smithi001.stdout:502: finishing rollback tid 0 to smithi00140323-1 2021-01-15T19:09:59.320 INFO:tasks.rados.rados.0.smithi001.stdout:502: finishing rollback tid 1 to smithi00140323-1 2021-01-15T19:09:59.321 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 136 (ObjNum 604025904 snap 32581 seq_num 603982032) dirty dne 2021-01-15T19:09:59.355 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:59.354+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:59.434 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:59.435 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:59.444 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:09:59.456 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:09:59.455+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:09:59.557 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:59.556+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:59.616 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:59.616 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:59.627 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:09:59.667 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:09:59.666+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:09:59.758 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:09:59.757+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:09:59.766 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:59.766 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:59.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:09:59.859 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:09:59.858+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:09:59.900 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:09:59.900 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:09:59.910 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:09:59.960 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:09:59.959+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:00.028 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:00.027+0000 7f3fb515a080 -1 osd.6 94 log_to_monitors {default=true} 2021-01-15T19:10:00.053 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:00.053 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:00.061 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:00.060+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:00.063 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:00.162 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:00.161+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:00.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:10:00.229 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:00.230 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:00.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:00.265 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:00.264+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:00.318 INFO:tasks.rados.rados.0.smithi001.stdout:502: done (1 left) 2021-01-15T19:10:00.319 INFO:tasks.rados.rados.0.smithi001.stdout:503: done (0 left) 2021-01-15T19:10:00.319 INFO:tasks.rados.rados.0.smithi001.stdout:504: copy_from oid 4 from oid 38 current snap is 58 2021-01-15T19:10:00.319 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 29 current snap is 58 2021-01-15T19:10:00.319 INFO:tasks.rados.rados.0.smithi001.stdout:505: seq_num 181 ranges {2629632=638976} 2021-01-15T19:10:00.336 INFO:tasks.rados.rados.0.smithi001.stdout:505: writing smithi00140323-29 from 2629632 to 3268608 tid 1 2021-01-15T19:10:00.337 INFO:tasks.rados.rados.0.smithi001.stdout:506: read oid 17 snap -1 2021-01-15T19:10:00.337 INFO:tasks.rados.rados.0.smithi001.stdout:506: expect (ObjNum 175 snap 56 seq_num 175) 2021-01-15T19:10:00.337 INFO:tasks.rados.rados.0.smithi001.stdout:507: delete oid 45 current snap is 58 2021-01-15T19:10:00.356 INFO:tasks.rados.rados.0.smithi001.stdout:505: finishing write tid 1 to smithi00140323-29 2021-01-15T19:10:00.356 INFO:tasks.rados.rados.0.smithi001.stdout:505: finishing write tid 2 to smithi00140323-29 2021-01-15T19:10:00.358 INFO:tasks.rados.rados.0.smithi001.stdout:506: expect (ObjNum 175 snap 56 seq_num 175) 2021-01-15T19:10:00.364 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:00.363+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:00.378 INFO:tasks.rados.rados.0.smithi001.stdout:505: finishing write tid 3 to smithi00140323-29 2021-01-15T19:10:00.378 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 140 (ObjNum 181 snap 58 seq_num 181) dirty exists 2021-01-15T19:10:00.378 INFO:tasks.rados.rados.0.smithi001.stdout:505: left oid 29 (ObjNum 181 snap 58 seq_num 181) 2021-01-15T19:10:00.379 INFO:tasks.rados.rados.0.smithi001.stdout:504: finishing copy_from to smithi00140323-4 2021-01-15T19:10:00.379 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 58 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:10:00.379 INFO:tasks.rados.rados.0.smithi001.stdout:504: finishing copy_from racing read to smithi00140323-4 2021-01-15T19:10:00.379 INFO:tasks.rados.rados.0.smithi001.stdout:504: done (3 left) 2021-01-15T19:10:00.380 INFO:tasks.rados.rados.0.smithi001.stdout:505: done (2 left) 2021-01-15T19:10:00.380 INFO:tasks.rados.rados.0.smithi001.stdout:506: done (1 left) 2021-01-15T19:10:00.380 INFO:tasks.rados.rados.0.smithi001.stdout:507: done (0 left) 2021-01-15T19:10:00.380 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 7 current snap is 58 2021-01-15T19:10:00.380 INFO:tasks.rados.rados.0.smithi001.stdout:508: seq_num 183 ranges {5472256=671744} 2021-01-15T19:10:00.393 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:00.393 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:00.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:00.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:00.416 INFO:tasks.rados.rados.0.smithi001.stdout:508: writing smithi00140323-7 from 5472256 to 6144000 tid 1 2021-01-15T19:10:00.416 INFO:tasks.rados.rados.0.smithi001.stdout:509: read oid 50 snap 38 2021-01-15T19:10:00.416 INFO:tasks.rados.rados.0.smithi001.stdout:509: expect (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:10:00.417 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 49 current snap is 58 2021-01-15T19:10:00.417 INFO:tasks.rados.rados.0.smithi001.stdout:510: seq_num 184 ranges {2056192=688128} 2021-01-15T19:10:00.427 INFO:tasks.rados.rados.0.smithi001.stdout:510: writing smithi00140323-49 from 2056192 to 2744320 tid 1 2021-01-15T19:10:00.430 INFO:tasks.rados.rados.0.smithi001.stdout:509: expect (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:10:00.464 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:00.463+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:00.489 INFO:tasks.rados.rados.0.smithi001.stdout:508: finishing write tid 1 to smithi00140323-7 2021-01-15T19:10:00.490 INFO:tasks.rados.rados.0.smithi001.stdout:508: finishing write tid 2 to smithi00140323-7 2021-01-15T19:10:00.490 INFO:tasks.rados.rados.0.smithi001.stdout:509: done (2 left) 2021-01-15T19:10:00.491 INFO:tasks.rados.rados.0.smithi001.stdout:511: read oid 31 snap 49 2021-01-15T19:10:00.491 INFO:tasks.rados.rados.0.smithi001.stdout:511: expect (ObjNum 159 snap 42 seq_num 159) 2021-01-15T19:10:00.491 INFO:tasks.rados.rados.0.smithi001.stdout:512: read oid 36 snap -1 2021-01-15T19:10:00.492 INFO:tasks.rados.rados.0.smithi001.stdout:512: expect (ObjNum 180 snap 56 seq_num 180) 2021-01-15T19:10:00.492 INFO:tasks.rados.rados.0.smithi001.stdout:513: rmattr oid 26 current snap is 58 2021-01-15T19:10:00.493 INFO:tasks.rados.rados.0.smithi001.stdout:508: finishing write tid 3 to smithi00140323-7 2021-01-15T19:10:00.493 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 193 (ObjNum 183 snap 58 seq_num 183) dirty exists 2021-01-15T19:10:00.493 INFO:tasks.rados.rados.0.smithi001.stdout:508: left oid 7 (ObjNum 183 snap 58 seq_num 183) 2021-01-15T19:10:00.494 INFO:tasks.rados.rados.0.smithi001.stdout:510: finishing write tid 1 to smithi00140323-49 2021-01-15T19:10:00.494 INFO:tasks.rados.rados.0.smithi001.stdout:510: finishing write tid 2 to smithi00140323-49 2021-01-15T19:10:00.495 INFO:tasks.rados.rados.0.smithi001.stdout:510: finishing write tid 3 to smithi00140323-49 2021-01-15T19:10:00.495 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 107 (ObjNum 184 snap 58 seq_num 184) dirty exists 2021-01-15T19:10:00.495 INFO:tasks.rados.rados.0.smithi001.stdout:510: left oid 49 (ObjNum 184 snap 58 seq_num 184) 2021-01-15T19:10:00.496 INFO:tasks.rados.rados.0.smithi001.stdout:508: done (4 left) 2021-01-15T19:10:00.496 INFO:tasks.rados.rados.0.smithi001.stdout:510: done (3 left) 2021-01-15T19:10:00.497 INFO:tasks.rados.rados.0.smithi001.stdout:514: copy_from oid 11 from oid 42 current snap is 58 2021-01-15T19:10:00.497 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 32 current snap is 58 2021-01-15T19:10:00.497 INFO:tasks.rados.rados.0.smithi001.stdout:515: seq_num 185 ranges {1941504=704512} 2021-01-15T19:10:00.508 INFO:tasks.rados.rados.0.smithi001.stdout:515: writing smithi00140323-32 from 1941504 to 2646016 tid 1 2021-01-15T19:10:00.509 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 80 (ObjNum 103 snap 17 seq_num 103) dirty exists 2021-01-15T19:10:00.509 INFO:tasks.rados.rados.0.smithi001.stdout:513: done (4 left) 2021-01-15T19:10:00.509 INFO:tasks.rados.rados.0.smithi001.stdout:516: snap_remove snap 36 2021-01-15T19:10:00.565 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:00.564+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:00.579 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:00.579 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:00.584 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:00.588 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:00.604 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:00.624 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:00.644 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:00.664 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:00.667 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:00.665+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:00.686 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:00.706 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:00.727 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:00.728 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.729 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.730 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:00.755 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:00.755 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:00.756 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:00.756 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:00.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:00.767 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:00.766+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:00.868 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:00.867+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:00.888 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:00.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:00.889 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:00.889 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:00.889 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:00.898 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:00.969 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:00.968+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:01.024 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:01.024 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:01.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:01.025 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:01.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.025 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:10:01.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:01.026 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 81.837520530000006, 2021-01-15T19:10:01.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:10:01.026 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.026 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.027 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:01.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:01.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:10:01.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:10:01.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:10:01.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:10:01.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:10:01.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:10:01.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:10:01.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:10:01.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:10:01.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:10:01.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:10:01.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:10:01.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:10:01.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:10:01.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:10:01.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:10:01.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:10:01.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.037 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.038 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=104, reqid=client.4418.0:434, at_version=51'40, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:10:01.038 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:10:01.038 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 81.756563080000006, 2021-01-15T19:10:01.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232, 2021-01-15T19:10:01.038 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.039 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.039 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:10:01.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.265679+0000", 2021-01-15T19:10:01.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000871673 2021-01-15T19:10:01.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266550+0000", 2021-01-15T19:10:01.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4765000000000001e-05 2021-01-15T19:10:01.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266575+0000", 2021-01-15T19:10:01.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2021-01-15T19:10:01.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266592+0000", 2021-01-15T19:10:01.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7000000000000003e-06 2021-01-15T19:10:01.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266602+0000", 2021-01-15T19:10:01.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3728999999999999e-05 2021-01-15T19:10:01.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266666+0000", 2021-01-15T19:10:01.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2744e-05 2021-01-15T19:10:01.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.266678+0000", 2021-01-15T19:10:01.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014243850000000001 2021-01-15T19:10:01.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280922+0000", 2021-01-15T19:10:01.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0741000000000001e-05 2021-01-15T19:10:01.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.280973+0000", 2021-01-15T19:10:01.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015294232 2021-01-15T19:10:01.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.049 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.050 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.050 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 51/21 ECSubWrite(tid=105, reqid=client.4418.0:435, at_version=51'41, trim_to=0'0, roll_forward_to=40'38))", 2021-01-15T19:10:01.050 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:10:01.051 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 81.752151806000001, 2021-01-15T19:10:01.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389, 2021-01-15T19:10:01.051 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.051 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.051 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:10:01.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270090+0000", 2021-01-15T19:10:01.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421900000000001 2021-01-15T19:10:01.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270264+0000", 2021-01-15T19:10:01.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3272999999999999e-05 2021-01-15T19:10:01.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270287+0000", 2021-01-15T19:10:01.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6549e-05 2021-01-15T19:10:01.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270304+0000", 2021-01-15T19:10:01.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.469e-06 2021-01-15T19:10:01.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.270312+0000", 2021-01-15T19:10:01.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033286829999999998 2021-01-15T19:10:01.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273640+0000", 2021-01-15T19:10:01.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6605000000000001e-05 2021-01-15T19:10:01.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.273657+0000", 2021-01-15T19:10:01.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0087196819999999994 2021-01-15T19:10:01.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282376+0000", 2021-01-15T19:10:01.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1909000000000001e-05 2021-01-15T19:10:01.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.282428+0000", 2021-01-15T19:10:01.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012338389 2021-01-15T19:10:01.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.064 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.065 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:01.065 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:01.065 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.790032353000001, 2021-01-15T19:10:01.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:10:01.066 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.066 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.067 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.067 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:10:01.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.067 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:01.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:01.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:10:01.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:10:01.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:10:01.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:10:01.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:10:01.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:10:01.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:01.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:10:01.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:10:01.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:10:01.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:10:01.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:10:01.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:10:01.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:10:01.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:10:01.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:10:01.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:10:01.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:01.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:10:01.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:10:01.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:10:01.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:10:01.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.080 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.081 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:10:01.081 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:01.081 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.789884334999996, 2021-01-15T19:10:01.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:10:01.081 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.082 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.082 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.082 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.082 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.082 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:10:01.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.083 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:01.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:01.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:10:01.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:10:01.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:10:01.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:10:01.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:10:01.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:10:01.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:10:01.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:10:01.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:10:01.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:10:01.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:10:01.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:01.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:10:01.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:10:01.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:10:01.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:10:01.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:01.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:10:01.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:10:01.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:10:01.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:10:01.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:10:01.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:10:01.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:10:01.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:10:01.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.096 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=201, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:10:01.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:10:01.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.669448817999999, 2021-01-15T19:10:01.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999, 2021-01-15T19:10:01.098 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.098 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.098 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:10:01.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352793+0000", 2021-01-15T19:10:01.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6137999999999998e-05 2021-01-15T19:10:01.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352839+0000", 2021-01-15T19:10:01.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2480000000000002e-06 2021-01-15T19:10:01.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352848+0000", 2021-01-15T19:10:01.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4456e-05 2021-01-15T19:10:01.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352863+0000", 2021-01-15T19:10:01.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.755e-06 2021-01-15T19:10:01.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352869+0000", 2021-01-15T19:10:01.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01049957 2021-01-15T19:10:01.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.363368+0000", 2021-01-15T19:10:01.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001221172 2021-01-15T19:10:01.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.364589+0000", 2021-01-15T19:10:01.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011796338999999999 2021-01-15T19:10:01.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.106 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.107 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=202, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:10:01.107 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:10:01.108 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.669333536000003, 2021-01-15T19:10:01.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293, 2021-01-15T19:10:01.108 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.108 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.108 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:10:01.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:10:01.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1519e-05 2021-01-15T19:10:01.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352950+0000", 2021-01-15T19:10:01.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3610000000000001e-06 2021-01-15T19:10:01.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352958+0000", 2021-01-15T19:10:01.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4105e-05 2021-01-15T19:10:01.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352972+0000", 2021-01-15T19:10:01.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3709999999999999e-06 2021-01-15T19:10:01.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352978+0000", 2021-01-15T19:10:01.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013549736999999999 2021-01-15T19:10:01.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.366527+0000", 2021-01-15T19:10:01.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025352 2021-01-15T19:10:01.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.369063+0000", 2021-01-15T19:10:01.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293 2021-01-15T19:10:01.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.117 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.117 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.117 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=257, reqid=client.4418.0:863, at_version=85'126, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:01.118 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:10:01.118 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.649890004, 2021-01-15T19:10:01.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972, 2021-01-15T19:10:01.118 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.118 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.119 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:10:01.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372352+0000", 2021-01-15T19:10:01.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000526152 2021-01-15T19:10:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372878+0000", 2021-01-15T19:10:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8391e-05 2021-01-15T19:10:01.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372906+0000", 2021-01-15T19:10:01.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2021-01-15T19:10:01.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372924+0000", 2021-01-15T19:10:01.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4420000000000001e-06 2021-01-15T19:10:01.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372931+0000", 2021-01-15T19:10:01.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040511399999999998 2021-01-15T19:10:01.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373336+0000", 2021-01-15T19:10:01.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4637000000000001e-05 2021-01-15T19:10:01.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.373351+0000", 2021-01-15T19:10:01.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014127954 2021-01-15T19:10:01.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387479+0000", 2021-01-15T19:10:01.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010297 2021-01-15T19:10:01.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.387582+0000", 2021-01-15T19:10:01.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015229972 2021-01-15T19:10:01.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.129 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.130 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=261, reqid=client.4418.0:849, at_version=85'130, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:01.130 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:10:01.130 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.622735163999998, 2021-01-15T19:10:01.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001, 2021-01-15T19:10:01.131 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.131 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.131 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:10:01.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:10:01.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041353340000000001 2021-01-15T19:10:01.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403642+0000", 2021-01-15T19:10:01.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6962999999999998e-05 2021-01-15T19:10:01.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403679+0000", 2021-01-15T19:10:01.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7178999999999999e-05 2021-01-15T19:10:01.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403696+0000", 2021-01-15T19:10:01.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:01.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403703+0000", 2021-01-15T19:10:01.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6183999999999997e-05 2021-01-15T19:10:01.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403749+0000", 2021-01-15T19:10:01.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5073e-05 2021-01-15T19:10:01.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403764+0000", 2021-01-15T19:10:01.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012428852000000001 2021-01-15T19:10:01.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416193+0000", 2021-01-15T19:10:01.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1882000000000002e-05 2021-01-15T19:10:01.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416245+0000", 2021-01-15T19:10:01.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001 2021-01-15T19:10:01.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.141 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.142 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:01.142 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:01.142 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.835563344000001, 2021-01-15T19:10:01.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:10:01.143 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.143 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.143 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.143 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.144 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.144 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:01.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.144 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:01.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:01.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:10:01.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:10:01.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:10:01.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:10:01.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:10:01.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:10:01.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:10:01.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:10:01.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:10:01.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:10:01.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:10:01.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:10:01.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:10:01.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:10:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:10:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:10:01.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:10:01.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:10:01.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:10:01.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.155 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.156 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:01.156 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:01.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.835372055000001, 2021-01-15T19:10:01.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:10:01.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.158 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:01.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.158 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:01.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:01.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:10:01.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:10:01.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:10:01.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:10:01.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:10:01.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:10:01.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:10:01.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:10:01.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:10:01.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:10:01.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:10:01.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:10:01.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:10:01.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:10:01.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:10:01.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:10:01.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:10:01.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:10:01.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:10:01.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:01.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:01.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.835209702, 2021-01-15T19:10:01.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:10:01.171 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.172 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:01.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:01.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:01.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:10:01.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:10:01.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:10:01.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:10:01.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:10:01.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:10:01.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:10:01.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:10:01.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:10:01.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:10:01.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:10:01.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:10:01.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:10:01.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:10:01.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:10:01.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:10:01.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:10:01.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:10:01.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:10:01.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.184 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.184 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.185 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:942 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~2203648,checksum] snapc 0=[] ondisk+read+known_if_redirected e90)", 2021-01-15T19:10:01.185 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:10:01.185 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.566478850000003, 2021-01-15T19:10:01.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817, 2021-01-15T19:10:01.186 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.186 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.187 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 942 2021-01-15T19:10:01.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.187 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:10:01.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455763+0000", 2021-01-15T19:10:01.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3491000000000006e-05 2021-01-15T19:10:01.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455826+0000", 2021-01-15T19:10:01.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1099999999999997e-06 2021-01-15T19:10:01.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455834+0000", 2021-01-15T19:10:01.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2245e-05 2021-01-15T19:10:01.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455846+0000", 2021-01-15T19:10:01.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1660000000000002e-06 2021-01-15T19:10:01.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.455851+0000", 2021-01-15T19:10:01.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001470865 2021-01-15T19:10:01.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457322+0000", 2021-01-15T19:10:01.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012032 2021-01-15T19:10:01.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.457442+0000", 2021-01-15T19:10:01.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097176199999999997 2021-01-15T19:10:01.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.467160+0000", 2021-01-15T19:10:01.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011396817 2021-01-15T19:10:01.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.196 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.197 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s1 91/90 ECSubReadReply(tid=71, attrs_read=0))", 2021-01-15T19:10:01.197 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:10:01.198 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.805954454999998, 2021-01-15T19:10:01.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001, 2021-01-15T19:10:01.198 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.198 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.198 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:10:01.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.216287+0000", 2021-01-15T19:10:01.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032454459999999999 2021-01-15T19:10:01.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219533+0000", 2021-01-15T19:10:01.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2258e-05 2021-01-15T19:10:01.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219545+0000", 2021-01-15T19:10:01.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3614999999999999e-05 2021-01-15T19:10:01.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219559+0000", 2021-01-15T19:10:01.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2020000000000001e-06 2021-01-15T19:10:01.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219565+0000", 2021-01-15T19:10:01.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4172999999999998e-05 2021-01-15T19:10:01.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.219619+0000", 2021-01-15T19:10:01.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098911899999999994 2021-01-15T19:10:01.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229510+0000", 2021-01-15T19:10:01.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013222884000000001 2021-01-15T19:10:01.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.207 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.207 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 96/92 ECSubWrite(tid=404, reqid=osd.4.0:404, at_version=96'68, trim_to=0'0, roll_forward_to=93'66))", 2021-01-15T19:10:01.208 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:10:01.208 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.810101938000003, 2021-01-15T19:10:01.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125, 2021-01-15T19:10:01.208 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.209 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.209 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:10:01.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212140+0000", 2021-01-15T19:10:01.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1780000000000002e-05 2021-01-15T19:10:01.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212192+0000", 2021-01-15T19:10:01.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2773e-05 2021-01-15T19:10:01.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212214+0000", 2021-01-15T19:10:01.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8345000000000001e-05 2021-01-15T19:10:01.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212233+0000", 2021-01-15T19:10:01.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8379999999999996e-06 2021-01-15T19:10:01.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212240+0000", 2021-01-15T19:10:01.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1579000000000002e-05 2021-01-15T19:10:01.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212291+0000", 2021-01-15T19:10:01.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3115999999999999e-05 2021-01-15T19:10:01.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.212304+0000", 2021-01-15T19:10:01.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011237044 2021-01-15T19:10:01.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223541+0000", 2021-01-15T19:10:01.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9774999999999999e-05 2021-01-15T19:10:01.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223571+0000", 2021-01-15T19:10:01.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01143125 2021-01-15T19:10:01.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.219 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.220 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:982 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:01.220 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:10:01.220 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.803550551999997, 2021-01-15T19:10:01.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641, 2021-01-15T19:10:01.221 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.221 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.222 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 982 2021-01-15T19:10:01.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.222 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:10:01.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:10:01.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8164999999999999e-05 2021-01-15T19:10:01.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218719+0000", 2021-01-15T19:10:01.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6679999999999999e-06 2021-01-15T19:10:01.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218723+0000", 2021-01-15T19:10:01.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0639999999999999e-06 2021-01-15T19:10:01.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218729+0000", 2021-01-15T19:10:01.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2021-01-15T19:10:01.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218732+0000", 2021-01-15T19:10:01.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033278300000000002 2021-01-15T19:10:01.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219064+0000", 2021-01-15T19:10:01.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118377 2021-01-15T19:10:01.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219183+0000", 2021-01-15T19:10:01.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014915026 2021-01-15T19:10:01.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.234098+0000", 2021-01-15T19:10:01.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641 2021-01-15T19:10:01.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.232 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.232 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:01.232 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:01.233 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.803489788999997, 2021-01-15T19:10:01.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:10:01.233 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.233 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.233 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.234 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.234 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.234 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:10:01.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.235 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:01.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:01.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:10:01.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:10:01.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:10:01.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:10:01.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:10:01.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:10:01.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:10:01.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:10:01.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:10:01.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:10:01.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:10:01.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:10:01.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:10:01.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:10:01.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:10:01.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:01.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:01.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.803428099999998, 2021-01-15T19:10:01.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:10:01.245 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.245 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:01.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:10:01.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.247 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:01.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:01.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:10:01.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:10:01.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:10:01.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:10:01.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:10:01.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:10:01.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:10:01.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:10:01.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:10:01.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:10:01.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:10:01.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:10:01.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:10:01.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:10:01.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:10:01.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.256 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.257 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 110/52 ECSubWrite(tid=513, reqid=client.4418.0:1096, at_version=110'199, trim_to=0'0, roll_forward_to=109'197))", 2021-01-15T19:10:01.257 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:10:01.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.804767553000001, 2021-01-15T19:10:01.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999, 2021-01-15T19:10:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:10:01.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:10:01.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091685599999999996 2021-01-15T19:10:01.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218391+0000", 2021-01-15T19:10:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6659999999999998e-05 2021-01-15T19:10:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218428+0000", 2021-01-15T19:10:01.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6130999999999999e-05 2021-01-15T19:10:01.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218444+0000", 2021-01-15T19:10:01.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-05 2021-01-15T19:10:01.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218455+0000", 2021-01-15T19:10:01.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0050000000000003e-05 2021-01-15T19:10:01.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218515+0000", 2021-01-15T19:10:01.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4779e-05 2021-01-15T19:10:01.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218530+0000", 2021-01-15T19:10:01.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014431424 2021-01-15T19:10:01.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.232961+0000", 2021-01-15T19:10:01.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.893e-05 2021-01-15T19:10:01.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233020+0000", 2021-01-15T19:10:01.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999 2021-01-15T19:10:01.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1142 3.9s0 3:935d2ce1:::smithi00140323-50:head [create,setxattr nOg\\b]PYmQpZprhrNSoOiXch_OVhaejNkYNj\\rrb (1507) in=40b,setxattr WcdVgSb^Xpei^gpqgVORjX_ifYYlebi^lg_\\ (1997) in=36b,setxattr aQpVtT`^]OieemiqNUnaUfic[c]Wqmdis`XQ_Tqm (908) in=40b,setxattr nYb]ZVbStbOagQf^W[sgda\\nc]R]dSaefWh]adhg (1893) in=40b,setxattr U^ja\\c]ahh_WgcSQkePloVfUV[_dZZa_Qa^jZsgO (1086) in=40b,setxattr TUgWsbVkpRdhTUXqjX]eOikS`Wa]sVkaNpWTYbY[ (1399) in=40b,setxattr gXqmVqsjhpeU\\nn\\SoQ`SZfYeg^]omdk_pVTP`do (806) in=40b,setxattr TrVPgo]^oWNoalZnotgZbhqliYh[id^Q_loQnh^Z (1963) in=40b,setxattr _TS_[m^pOge]N^c\\[mgPssoXoqlONURs`Shj\\UYk (1647) in=40b,setxattr W^YiocPQgkgoftgn`q_YOW_UUYQpSd_s[^sOQfqh (874) in=40b,setxattr lbeYtWUYqpORgbP`aWmV\\TSo[odhaTnocqnTrlQr (945) in=40b,setxattr QdO^gYZc\\rqThdOUbdYlmofVstjYkpQs_ab]o_jT (1786) in=40b,setxattr hNj]sQWrnXhXNh[PPa`V^jVe_RrNYUN^^^\\VmZj[ (943) in=40b,setxattr s_fmbQYrifmdiN_RPaXpdaZd[\\WffOb^[VNTtQjW (383) in=40b,setxattr _kZU^ZanaPRW_XWjbYlgtXRWi^T[O^TVpkTs`npQ (1609) in=40b,setxattr qaNdNbjbVYZU[Ygrejofh`kRTiOS_jZ_PkYZ^U[p (733) in=40b,setxattr i_NbbQjhTiYWVccoc`[glilPmsbWr]RalgkhRmcc (417) in=40b,setxattr aZess_ORomNedc_mbO_QcQqabPbRt_aa\\dslZoPr (17332) in=40b,setxattr Roai^sie^qdaa[nObntZWmUWiV]flrp`qaaVRecS (1785) in=40b,setxattr ek^QaVgeZSSY_ck^tsPa^ncnge]eT_asrQ_NOa\\Q (1153) in=40b] snapc 5c=[5c,58,57,56,53,4b,44,41,26] ondisk+write+known_if_redirected e119)", 2021-01-15T19:10:01.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:10:01.270 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.742916169000001, 2021-01-15T19:10:01.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875, 2021-01-15T19:10:01.270 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:01.270 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:01.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:01.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.271 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1142 2021-01-15T19:10:01.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.271 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:01.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:01.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:10:01.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:01.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:01.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279326+0000", 2021-01-15T19:10:01.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4063999999999999e-05 2021-01-15T19:10:01.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:01.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279420+0000", 2021-01-15T19:10:01.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9740000000000002e-06 2021-01-15T19:10:01.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:01.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279428+0000", 2021-01-15T19:10:01.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3764999999999999e-05 2021-01-15T19:10:01.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:01.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279441+0000", 2021-01-15T19:10:01.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4460000000000004e-06 2021-01-15T19:10:01.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279449+0000", 2021-01-15T19:10:01.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5884000000000003e-05 2021-01-15T19:10:01.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:01.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279495+0000", 2021-01-15T19:10:01.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014788 2021-01-15T19:10:01.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:01.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.279643+0000", 2021-01-15T19:10:01.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104872 2021-01-15T19:10:01.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.280748+0000", 2021-01-15T19:10:01.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044365569999999998 2021-01-15T19:10:01.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.285184+0000", 2021-01-15T19:10:01.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064052199999999997 2021-01-15T19:10:01.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:01.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291589+0000", 2021-01-15T19:10:01.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4213000000000002e-05 2021-01-15T19:10:01.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:01.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:01.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:01.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:47.291664+0000", 2021-01-15T19:10:01.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012337875 2021-01-15T19:10:01.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:01.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:01.284 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:01.286 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:01.285+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:01.286 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:01.323 INFO:tasks.rados.rados.0.smithi001.stdout:516: done (4 left) 2021-01-15T19:10:01.323 INFO:tasks.rados.rados.0.smithi001.stdout:517: rmattr oid 13 current snap is 58 2021-01-15T19:10:01.323 INFO:tasks.rados.rados.0.smithi001.stdout:517: done (4 left) 2021-01-15T19:10:01.324 INFO:tasks.rados.rados.0.smithi001.stdout:518: copy_from oid 23 from oid 29 current snap is 58 2021-01-15T19:10:01.324 INFO:tasks.rados.rados.0.smithi001.stdout:511: expect (ObjNum 159 snap 42 seq_num 159) 2021-01-15T19:10:01.344 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:10:01.387 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:01.386+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:01.445 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:01.445 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:01.446 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:01.446 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:01.455 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:01.488 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:01.487+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:01.502 INFO:tasks.rados.rados.0.smithi001.stdout:514: finishing copy_from to smithi00140323-11 2021-01-15T19:10:01.502 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 52 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:10:01.503 INFO:tasks.rados.rados.0.smithi001.stdout:512: expect (ObjNum 180 snap 56 seq_num 180) 2021-01-15T19:10:01.588 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:01.588 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:01.588 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:01.589 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:01.589 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:01.589 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:01.587+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:01.597 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:01.667 INFO:tasks.rados.rados.0.smithi001.stdout:514: finishing copy_from racing read to smithi00140323-11 2021-01-15T19:10:01.668 INFO:tasks.rados.rados.0.smithi001.stdout:515: finishing write tid 1 to smithi00140323-32 2021-01-15T19:10:01.668 INFO:tasks.rados.rados.0.smithi001.stdout:515: finishing write tid 2 to smithi00140323-32 2021-01-15T19:10:01.668 INFO:tasks.rados.rados.0.smithi001.stdout:515: finishing write tid 3 to smithi00140323-32 2021-01-15T19:10:01.668 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 110 (ObjNum 185 snap 58 seq_num 185) dirty exists 2021-01-15T19:10:01.668 INFO:tasks.rados.rados.0.smithi001.stdout:515: left oid 32 (ObjNum 185 snap 58 seq_num 185) 2021-01-15T19:10:01.669 INFO:tasks.rados.rados.0.smithi001.stdout:511: done (4 left) 2021-01-15T19:10:01.669 INFO:tasks.rados.rados.0.smithi001.stdout:512: done (3 left) 2021-01-15T19:10:01.669 INFO:tasks.rados.rados.0.smithi001.stdout:514: done (2 left) 2021-01-15T19:10:01.669 INFO:tasks.rados.rados.0.smithi001.stdout:515: done (1 left) 2021-01-15T19:10:01.670 INFO:tasks.rados.rados.0.smithi001.stdout:519: read oid 27 snap 56 2021-01-15T19:10:01.670 INFO:tasks.rados.rados.0.smithi001.stdout:519: expect deleted 2021-01-15T19:10:01.670 INFO:tasks.rados.rados.0.smithi001.stdout:520: read oid 49 snap -1 2021-01-15T19:10:01.670 INFO:tasks.rados.rados.0.smithi001.stdout:520: expect (ObjNum 184 snap 58 seq_num 184) 2021-01-15T19:10:01.670 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 21 current snap is 58 2021-01-15T19:10:01.671 INFO:tasks.rados.rados.0.smithi001.stdout:518: finishing copy_from to smithi00140323-23 2021-01-15T19:10:01.671 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 23 v 140 (ObjNum 181 snap 58 seq_num 181) dirty exists 2021-01-15T19:10:01.671 INFO:tasks.rados.rados.0.smithi001.stdout:518: finishing copy_from racing read to smithi00140323-23 2021-01-15T19:10:01.671 INFO:tasks.rados.rados.0.smithi001.stdout:521: seq_num 186 ranges {2949120=720896} 2021-01-15T19:10:01.688 INFO:tasks.rados.rados.0.smithi001.stdout:521: writing smithi00140323-21 from 2949120 to 3670016 tid 1 2021-01-15T19:10:01.688 INFO:tasks.rados.rados.0.smithi001.stdout:518: done (3 left) 2021-01-15T19:10:01.688 INFO:tasks.rados.rados.0.smithi001.stdout:522: read oid 18 snap -1 2021-01-15T19:10:01.689 INFO:tasks.rados.rados.0.smithi001.stdout:522: expect deleted 2021-01-15T19:10:01.689 INFO:tasks.rados.rados.0.smithi001.stdout:523: read oid 35 snap -1 2021-01-15T19:10:01.689 INFO:tasks.rados.rados.0.smithi001.stdout:523: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:10:01.689 INFO:tasks.rados.rados.0.smithi001.stdout:524: rollback oid 45 current snap is 58 2021-01-15T19:10:01.690 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 45 to 52 2021-01-15T19:10:01.690 INFO:tasks.rados.rados.0.smithi001.stdout:520: expect (ObjNum 184 snap 58 seq_num 184) 2021-01-15T19:10:01.694 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:01.692+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:01.789 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:01.789 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:01.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:01.789 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:01.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:434 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~663552] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:01.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:10:01.790 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.526993234000003, 2021-01-15T19:10:01.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022929644999999999, 2021-01-15T19:10:01.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 434 2021-01-15T19:10:01.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.792 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:10:01.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258548+0000", 2021-01-15T19:10:01.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044819800000000002 2021-01-15T19:10:01.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.258996+0000", 2021-01-15T19:10:01.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4090000000000001e-05 2021-01-15T19:10:01.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259011+0000", 2021-01-15T19:10:01.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7579000000000001e-05 2021-01-15T19:10:01.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259038+0000", 2021-01-15T19:10:01.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1133e-05 2021-01-15T19:10:01.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259049+0000", 2021-01-15T19:10:01.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3312000000000006e-05 2021-01-15T19:10:01.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259123+0000", 2021-01-15T19:10:01.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3225e-05 2021-01-15T19:10:01.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:01.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.259136+0000", 2021-01-15T19:10:01.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002543362 2021-01-15T19:10:01.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261679+0000", 2021-01-15T19:10:01.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012904299999999999 2021-01-15T19:10:01.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261808+0000", 2021-01-15T19:10:01.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001322642 2021-01-15T19:10:01.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.263131+0000", 2021-01-15T19:10:01.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012144998000000001 2021-01-15T19:10:01.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275276+0000", 2021-01-15T19:10:01.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060959580000000003 2021-01-15T19:10:01.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.281372+0000", 2021-01-15T19:10:01.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000106105 2021-01-15T19:10:01.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.281478+0000", 2021-01-15T19:10:01.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022929644999999999 2021-01-15T19:10:01.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.806 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.807 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:01.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:01.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.532279721999998, 2021-01-15T19:10:01.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:01.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:01.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:01.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:01.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:01.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:01.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:01.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:01.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:01.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:01.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:01.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:01.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:01.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:01.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:01.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:01.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:01.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:01.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:01.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:01.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:01.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:01.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:01.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:01.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:01.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.823 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:01.823 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:01.823 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.532114609000004, 2021-01-15T19:10:01.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:01.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:01.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:01.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.825 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:01.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:01.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:01.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:01.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:01.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:01.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:01.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:01.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:01.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:01.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:01.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:01.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:01.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:01.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:01.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:01.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:01.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:01.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:01.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:01.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:01.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:01.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:01.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:01.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:01.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:01.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:01.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.839 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:01.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:01.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.524834560000002, 2021-01-15T19:10:01.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:01.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:01.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.842 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:01.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:01.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:01.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:01.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:01.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:01.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:01.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:01.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:01.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:01.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:01.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:01.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:01.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:01.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:01.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:01.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:01.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:01.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:01.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:01.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:01.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:01.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:01.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.855 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:01.855 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:01.856 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.524707927000001, 2021-01-15T19:10:01.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:01.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:01.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:01.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.857 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:01.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:01.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:01.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:01.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:01.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:01.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:01.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:01.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:01.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:01.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:01.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:01.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:01.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:01.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:01.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:01.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:01.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:01.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:01.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:01.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:01.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:01.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:01.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:01.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:01.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:01.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:01.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:01.873 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:01.873 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.441970961999999, 2021-01-15T19:10:01.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:01.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.874 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.874 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.874 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:01.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.875 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:01.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:01.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:01.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:01.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:01.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:01.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:01.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:01.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:01.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:01.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:01.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:01.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:01.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:01.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:01.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:01.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:01.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:01.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:01.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:01.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:01.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:01.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:01.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.888 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:01.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:01.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.441821347999998, 2021-01-15T19:10:01.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:01.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:01.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.421523665999999, 2021-01-15T19:10:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:01.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:01.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:01.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:01.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:01.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:01.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:01.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:01.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:01.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:01.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:01.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:01.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:01.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:01.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:01.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:01.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:01.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:01.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:01.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:01.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:01.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.930 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:01.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:01.931 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.418467014999997, 2021-01-15T19:10:01.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:01.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.932 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:01.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.933 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:01.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:01.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:01.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:01.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:01.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:01.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:01.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:01.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:01.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:01.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:01.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:01.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:01.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:01.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:01.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:01.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:01.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:01.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:01.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:01.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:01.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:01.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:01.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:01.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:01.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:01.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:01.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.952 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:01.952 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:01.952 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.415453753000001, 2021-01-15T19:10:01.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:01.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:01.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.955 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:01.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:01.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:01.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:01.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:01.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:01.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:01.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:01.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:01.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:01.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:01.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:01.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:01.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:01.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:01.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:01.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:01.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:01.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:01.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:01.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:01.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:01.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:01.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:01.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:01.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:01.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:01.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.974 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.975 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:01.975 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:01.975 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.412706141999998, 2021-01-15T19:10:01.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:01.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.976 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:01.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.978 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:01.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:01.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:01.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:01.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:01.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:01.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:01.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:01.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:01.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:01.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:01.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:01.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:01.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:01.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:01.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:01.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:01.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:01.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:01.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:01.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:01.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:01.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:01.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:01.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:01.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:01.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:01.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:01.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:01.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:01.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:01.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:01.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:01.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:01.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:01.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:01.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:01.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:01.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:01.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:01.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:01.995 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.995 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:01.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:01.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.996 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:01.997 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:01.997 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.41227232, 2021-01-15T19:10:01.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:01.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:01.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:01.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:01.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:01.998 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:01.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:01.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:01.999 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:01.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:01.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:02.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:02.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:02.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:02.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:02.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:02.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:02.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:02.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:02.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:02.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:02.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:02.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:02.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:02.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:02.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:02.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:02.018 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.412148160999998, 2021-01-15T19:10:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.019 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.020 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:02.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:02.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:02.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:02.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:02.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:02.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:02.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:02.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.307471626000002, 2021-01-15T19:10:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:02.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:02.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:02.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:02.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:02.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:02.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:02.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:02.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:02.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:02.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:02.065 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.427250547, 2021-01-15T19:10:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.066 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:02.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:02.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:02.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:02.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:02.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:02.079 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.397532390000002, 2021-01-15T19:10:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:02.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:02.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:02.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.090 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.091 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.393446222999998, 2021-01-15T19:10:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:02.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:02.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:02.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:02.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:02.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:02.104 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.393310472000003, 2021-01-15T19:10:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.105 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=28'16, committed=1, applied=1))", 2021-01-15T19:10:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:10:02.115 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.393083623000003, 2021-01-15T19:10:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022959151000000001, 2021-01-15T19:10:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:10:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:10:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6398999999999997e-05 2021-01-15T19:10:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392544+0000", 2021-01-15T19:10:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1029999999999999e-05 2021-01-15T19:10:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392555+0000", 2021-01-15T19:10:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4134999999999999e-05 2021-01-15T19:10:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392569+0000", 2021-01-15T19:10:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.874e-06 2021-01-15T19:10:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392575+0000", 2021-01-15T19:10:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022133907000000001 2021-01-15T19:10:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414709+0000", 2021-01-15T19:10:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070780599999999995 2021-01-15T19:10:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415417+0000", 2021-01-15T19:10:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022959151000000001 2021-01-15T19:10:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:02.125 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.5548114890000004, 2021-01-15T19:10:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:02.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:02.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:02.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:02.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:02.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:02.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:02.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:02.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:02.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:02.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.140 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:02.141 INFO:tasks.rados.rados.0.smithi001.stdout:524: finishing rollback tid 1 to smithi00140323-45 2021-01-15T19:10:02.142 INFO:tasks.rados.rados.0.smithi001.stdout:524: finishing rollback tid 2 to smithi00140323-45 2021-01-15T19:10:02.142 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 197 (ObjNum 89 snap 16 seq_num 89) dirty exists 2021-01-15T19:10:02.142 INFO:tasks.rados.rados.0.smithi001.stdout:519: done (5 left) 2021-01-15T19:10:02.142 INFO:tasks.rados.rados.0.smithi001.stdout:520: done (4 left) 2021-01-15T19:10:02.143 INFO:tasks.rados.rados.0.smithi001.stdout:522: done (3 left) 2021-01-15T19:10:02.143 INFO:tasks.rados.rados.0.smithi001.stdout:524: done (2 left) 2021-01-15T19:10:02.143 INFO:tasks.rados.rados.0.smithi001.stdout:525: rollback oid 50 current snap is 58 2021-01-15T19:10:02.143 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 50 to 40 2021-01-15T19:10:02.143 INFO:tasks.rados.rados.0.smithi001.stdout:523: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:10:02.144 INFO:tasks.rados.rados.0.smithi001.stdout:521: finishing write tid 1 to smithi00140323-21 2021-01-15T19:10:02.144 INFO:tasks.rados.rados.0.smithi001.stdout:521: finishing write tid 2 to smithi00140323-21 2021-01-15T19:10:02.144 INFO:tasks.rados.rados.0.smithi001.stdout:521: finishing write tid 3 to smithi00140323-21 2021-01-15T19:10:02.144 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 85 (ObjNum 186 snap 58 seq_num 186) dirty exists 2021-01-15T19:10:02.144 INFO:tasks.rados.rados.0.smithi001.stdout:521: left oid 21 (ObjNum 186 snap 58 seq_num 186) 2021-01-15T19:10:02.145 INFO:tasks.rados.rados.0.smithi001.stdout:525: finishing rollback tid 0 to smithi00140323-50 2021-01-15T19:10:02.145 INFO:tasks.rados.rados.0.smithi001.stdout:521: done (2 left) 2021-01-15T19:10:02.145 INFO:tasks.rados.rados.0.smithi001.stdout:523: done (1 left) 2021-01-15T19:10:02.145 INFO:tasks.rados.rados.0.smithi001.stdout:526: snap_remove snap 49 2021-01-15T19:10:02.146 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:02.140+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:02.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:10:02.242 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:02.241+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:02.326 INFO:tasks.rados.rados.0.smithi001.stdout:526: done (1 left) 2021-01-15T19:10:02.326 INFO:tasks.rados.rados.0.smithi001.stdout:527: read oid 22 snap -1 2021-01-15T19:10:02.326 INFO:tasks.rados.rados.0.smithi001.stdout:527: expect (ObjNum 158 snap 41 seq_num 158) 2021-01-15T19:10:02.327 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 31 current snap is 58 2021-01-15T19:10:02.327 INFO:tasks.rados.rados.0.smithi001.stdout:525: finishing rollback tid 1 to smithi00140323-50 2021-01-15T19:10:02.327 INFO:tasks.rados.rados.0.smithi001.stdout:528: seq_num 187 ranges {2973696=737280} 2021-01-15T19:10:02.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:10:02.343 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:02.342+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:02.347 INFO:tasks.rados.rados.0.smithi001.stdout:528: writing smithi00140323-31 from 2973696 to 3710976 tid 1 2021-01-15T19:10:02.351 INFO:tasks.rados.rados.0.smithi001.stdout:525: finishing rollback tid 2 to smithi00140323-50 2021-01-15T19:10:02.351 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 143 (ObjNum 53 snap 1 seq_num 53) dirty exists 2021-01-15T19:10:02.352 INFO:tasks.rados.rados.0.smithi001.stdout:527: expect (ObjNum 158 snap 41 seq_num 158) 2021-01-15T19:10:02.388 INFO:tasks.rados.rados.0.smithi001.stdout:525: done (2 left) 2021-01-15T19:10:02.388 INFO:tasks.rados.rados.0.smithi001.stdout:527: done (1 left) 2021-01-15T19:10:02.389 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 8 current snap is 58 2021-01-15T19:10:02.389 INFO:tasks.rados.rados.0.smithi001.stdout:528: finishing write tid 1 to smithi00140323-31 2021-01-15T19:10:02.389 INFO:tasks.rados.rados.0.smithi001.stdout:528: finishing write tid 2 to smithi00140323-31 2021-01-15T19:10:02.389 INFO:tasks.rados.rados.0.smithi001.stdout:528: finishing write tid 3 to smithi00140323-31 2021-01-15T19:10:02.389 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 90 (ObjNum 187 snap 58 seq_num 187) dirty exists 2021-01-15T19:10:02.390 INFO:tasks.rados.rados.0.smithi001.stdout:528: left oid 31 (ObjNum 187 snap 58 seq_num 187) 2021-01-15T19:10:02.390 INFO:tasks.rados.rados.0.smithi001.stdout:529: seq_num 188 ranges {2203648=753664} 2021-01-15T19:10:02.404 INFO:tasks.rados.rados.0.smithi001.stdout:529: writing smithi00140323-8 from 2203648 to 2957312 tid 1 2021-01-15T19:10:02.404 INFO:tasks.rados.rados.0.smithi001.stdout:528: done (1 left) 2021-01-15T19:10:02.404 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 10 current snap is 58 2021-01-15T19:10:02.405 INFO:tasks.rados.rados.0.smithi001.stdout:530: seq_num 189 ranges {3186688=770048} 2021-01-15T19:10:02.426 INFO:tasks.rados.rados.0.smithi001.stdout:530: writing smithi00140323-10 from 3186688 to 3956736 tid 1 2021-01-15T19:10:02.426 INFO:tasks.rados.rados.0.smithi001.stdout:531: read oid 7 snap 55 2021-01-15T19:10:02.426 INFO:tasks.rados.rados.0.smithi001.stdout:531: expect (ObjNum 173 snap 52 seq_num 173) 2021-01-15T19:10:02.427 INFO:tasks.rados.rados.0.smithi001.stdout:532: read oid 3 snap 45 2021-01-15T19:10:02.427 INFO:tasks.rados.rados.0.smithi001.stdout:532: expect deleted 2021-01-15T19:10:02.427 INFO:tasks.rados.rados.0.smithi001.stdout:533: snap_remove snap 47 2021-01-15T19:10:02.443 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:02.442+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:02.497 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:02.497 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:02.498 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:02.507 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:10:02.548 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:02.547+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:02.646 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:02.645+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:02.681 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:02.681 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:02.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:02.682 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:02.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.682 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:10:02.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:02.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 132.16185369499999, 2021-01-15T19:10:02.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:10:02.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:10:02.684 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:10:02.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:10:02.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.684 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:02.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:02.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:10:02.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:10:02.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:10:02.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:10:02.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:10:02.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:10:02.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:10:02.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:10:02.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:10:02.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:10:02.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:10:02.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:02.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:10:02.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:10:02.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:10:02.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:10:02.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:10:02.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:10:02.695 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.695 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.695 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.696 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:828 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:02.696 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:10:02.696 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.496213148999999, 2021-01-15T19:10:02.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011908975, 2021-01-15T19:10:02.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 828 2021-01-15T19:10:02.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.698 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:10:02.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:10:02.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9705000000000001e-05 2021-01-15T19:10:02.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182525+0000", 2021-01-15T19:10:02.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6079999999999998e-06 2021-01-15T19:10:02.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182529+0000", 2021-01-15T19:10:02.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9899999999999997e-06 2021-01-15T19:10:02.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182537+0000", 2021-01-15T19:10:02.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1290000000000004e-06 2021-01-15T19:10:02.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182542+0000", 2021-01-15T19:10:02.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2103999999999999e-05 2021-01-15T19:10:02.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182615+0000", 2021-01-15T19:10:02.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0329999999999992e-06 2021-01-15T19:10:02.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:02.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:02.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040604489999999998 2021-01-15T19:10:02.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.186683+0000", 2021-01-15T19:10:02.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116184 2021-01-15T19:10:02.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.186799+0000", 2021-01-15T19:10:02.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0076047729999999996 2021-01-15T19:10:02.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.194404+0000", 2021-01-15T19:10:02.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011908975 2021-01-15T19:10:02.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.710 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:829 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:02.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:02.711 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.496139374999998, 2021-01-15T19:10:02.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001, 2021-01-15T19:10:02.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 829 2021-01-15T19:10:02.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.713 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:02.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:02.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4360000000000001e-05 2021-01-15T19:10:02.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:02.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0169999999999999e-06 2021-01-15T19:10:02.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182630+0000", 2021-01-15T19:10:02.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4198e-05 2021-01-15T19:10:02.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182644+0000", 2021-01-15T19:10:02.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1253999999999999e-05 2021-01-15T19:10:02.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182656+0000", 2021-01-15T19:10:02.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9756999999999999e-05 2021-01-15T19:10:02.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182695+0000", 2021-01-15T19:10:02.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2819999999999996e-06 2021-01-15T19:10:02.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:02.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182704+0000", 2021-01-15T19:10:02.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048051730000000003 2021-01-15T19:10:02.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187509+0000", 2021-01-15T19:10:02.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114002 2021-01-15T19:10:02.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187623+0000", 2021-01-15T19:10:02.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012123558 2021-01-15T19:10:02.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.199746+0000", 2021-01-15T19:10:02.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001 2021-01-15T19:10:02.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.725 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=315, reqid=client.4418.0:833, at_version=80'84, trim_to=0'0, roll_forward_to=78'82))", 2021-01-15T19:10:02.725 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:02.725 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.491577407999998, 2021-01-15T19:10:02.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001, 2021-01-15T19:10:02.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.726 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:02.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:02.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6477000000000002e-05 2021-01-15T19:10:02.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187187+0000", 2021-01-15T19:10:02.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4749e-05 2021-01-15T19:10:02.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187212+0000", 2021-01-15T19:10:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5977e-05 2021-01-15T19:10:02.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187228+0000", 2021-01-15T19:10:02.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0770000000000008e-06 2021-01-15T19:10:02.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187236+0000", 2021-01-15T19:10:02.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014010800000000001 2021-01-15T19:10:02.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187376+0000", 2021-01-15T19:10:02.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8624999999999999e-05 2021-01-15T19:10:02.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187395+0000", 2021-01-15T19:10:02.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013611689 2021-01-15T19:10:02.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201006+0000", 2021-01-15T19:10:02.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7980999999999999e-05 2021-01-15T19:10:02.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201064+0000", 2021-01-15T19:10:02.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001 2021-01-15T19:10:02.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.737 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.738 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:971 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:02.739 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:02.739 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.469434014000001, 2021-01-15T19:10:02.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002, 2021-01-15T19:10:02.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.740 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 971 2021-01-15T19:10:02.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.741 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:02.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:02.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2442999999999998e-05 2021-01-15T19:10:02.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209337+0000", 2021-01-15T19:10:02.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0950000000000002e-06 2021-01-15T19:10:02.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209346+0000", 2021-01-15T19:10:02.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9199999999999999e-05 2021-01-15T19:10:02.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209365+0000", 2021-01-15T19:10:02.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5750000000000007e-06 2021-01-15T19:10:02.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209374+0000", 2021-01-15T19:10:02.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074262999999999998 2021-01-15T19:10:02.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210117+0000", 2021-01-15T19:10:02.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022372900000000001 2021-01-15T19:10:02.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210341+0000", 2021-01-15T19:10:02.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016740338 2021-01-15T19:10:02.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227081+0000", 2021-01-15T19:10:02.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002 2021-01-15T19:10:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:972 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.469287258000001, 2021-01-15T19:10:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999, 2021-01-15T19:10:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 972 2021-01-15T19:10:02.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:02.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:02.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5875000000000003e-05 2021-01-15T19:10:02.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209477+0000", 2021-01-15T19:10:02.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7499999999999992e-06 2021-01-15T19:10:02.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209486+0000", 2021-01-15T19:10:02.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4825e-05 2021-01-15T19:10:02.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209500+0000", 2021-01-15T19:10:02.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:02.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209508+0000", 2021-01-15T19:10:02.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012352109999999999 2021-01-15T19:10:02.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210743+0000", 2021-01-15T19:10:02.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012467700000000001 2021-01-15T19:10:02.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210867+0000", 2021-01-15T19:10:02.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018560825 2021-01-15T19:10:02.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.229428+0000", 2021-01-15T19:10:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999 2021-01-15T19:10:02.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.763 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:973 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:02.764 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:02.764 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.469155925999999, 2021-01-15T19:10:02.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001, 2021-01-15T19:10:02.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 973 2021-01-15T19:10:02.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.766 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:02.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:02.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7021999999999997e-05 2021-01-15T19:10:02.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209609+0000", 2021-01-15T19:10:02.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3140000000000004e-06 2021-01-15T19:10:02.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209618+0000", 2021-01-15T19:10:02.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4259e-05 2021-01-15T19:10:02.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209632+0000", 2021-01-15T19:10:02.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2021-01-15T19:10:02.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209639+0000", 2021-01-15T19:10:02.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039100000000000003 2021-01-15T19:10:02.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213549+0000", 2021-01-15T19:10:02.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124948 2021-01-15T19:10:02.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213674+0000", 2021-01-15T19:10:02.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018190644999999998 2021-01-15T19:10:02.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.231864+0000", 2021-01-15T19:10:02.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001 2021-01-15T19:10:02.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1093 3.10s0 3:095f0b32:::smithi00140323-47:head [rollback 50] snapc 50=[50,4b,44,41,26] ondisk+write+known_if_redirected e109)", 2021-01-15T19:10:02.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:02.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.442481792999999, 2021-01-15T19:10:02.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242, 2021-01-15T19:10:02.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.777 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1093 2021-01-15T19:10:02.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.778 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:02.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:02.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3699999999999998e-05 2021-01-15T19:10:02.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236270+0000", 2021-01-15T19:10:02.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4700000000000008e-06 2021-01-15T19:10:02.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236280+0000", 2021-01-15T19:10:02.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6716999999999999e-05 2021-01-15T19:10:02.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236296+0000", 2021-01-15T19:10:02.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5490000000000006e-06 2021-01-15T19:10:02.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236305+0000", 2021-01-15T19:10:02.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4991999999999998e-05 2021-01-15T19:10:02.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236360+0000", 2021-01-15T19:10:02.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3660000000000003e-06 2021-01-15T19:10:02.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:02.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236367+0000", 2021-01-15T19:10:02.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005477549 2021-01-15T19:10:02.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241845+0000", 2021-01-15T19:10:02.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137458 2021-01-15T19:10:02.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241982+0000", 2021-01-15T19:10:02.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097224410000000001 2021-01-15T19:10:02.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.251705+0000", 2021-01-15T19:10:02.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242 2021-01-15T19:10:02.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 109/95 ECSubWrite(tid=263, reqid=client.4418.0:1090, at_version=109'60, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:02.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:02.790 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.438666155, 2021-01-15T19:10:02.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999, 2021-01-15T19:10:02.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:02.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:02.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8936e-05 2021-01-15T19:10:02.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240091+0000", 2021-01-15T19:10:02.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1778000000000001e-05 2021-01-15T19:10:02.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240113+0000", 2021-01-15T19:10:02.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6714999999999998e-05 2021-01-15T19:10:02.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240129+0000", 2021-01-15T19:10:02.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:02.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240137+0000", 2021-01-15T19:10:02.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9104999999999998e-05 2021-01-15T19:10:02.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240186+0000", 2021-01-15T19:10:02.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:02.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240194+0000", 2021-01-15T19:10:02.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017479262999999998 2021-01-15T19:10:02.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257674+0000", 2021-01-15T19:10:02.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9951999999999999e-05 2021-01-15T19:10:02.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257724+0000", 2021-01-15T19:10:02.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999 2021-01-15T19:10:02.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.802 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 110/95 ECSubWrite(tid=269, reqid=client.4418.0:1098, at_version=110'64, trim_to=0'0, roll_forward_to=109'62))", 2021-01-15T19:10:02.802 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:02.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.484605145, 2021-01-15T19:10:02.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999, 2021-01-15T19:10:02.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.803 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:02.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5725e-05 2021-01-15T19:10:02.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194149+0000", 2021-01-15T19:10:02.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3603000000000003e-05 2021-01-15T19:10:02.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194182+0000", 2021-01-15T19:10:02.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0721e-05 2021-01-15T19:10:02.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194203+0000", 2021-01-15T19:10:02.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3929999999999997e-06 2021-01-15T19:10:02.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194212+0000", 2021-01-15T19:10:02.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0593000000000006e-05 2021-01-15T19:10:02.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194293+0000", 2021-01-15T19:10:02.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3596e-05 2021-01-15T19:10:02.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194307+0000", 2021-01-15T19:10:02.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017420983000000001 2021-01-15T19:10:02.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211728+0000", 2021-01-15T19:10:02.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6654999999999999e-05 2021-01-15T19:10:02.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211784+0000", 2021-01-15T19:10:02.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999 2021-01-15T19:10:02.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.814 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.815 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1116 3.4s1 3:21d37027:::smithi00140323-27:head [copy-from ver 54] snapc 57=[57,56,53,50,4b,44,41,26] ondisk+write+known_if_redirected e114)", 2021-01-15T19:10:02.815 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:02.815 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.426166543000001, 2021-01-15T19:10:02.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999, 2021-01-15T19:10:02.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:02.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1116 2021-01-15T19:10:02.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.817 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:02.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:02.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3304000000000002e-05 2021-01-15T19:10:02.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252605+0000", 2021-01-15T19:10:02.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0723000000000001e-05 2021-01-15T19:10:02.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252616+0000", 2021-01-15T19:10:02.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0720000000000002e-05 2021-01-15T19:10:02.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252636+0000", 2021-01-15T19:10:02.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1799e-05 2021-01-15T19:10:02.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252648+0000", 2021-01-15T19:10:02.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5387000000000003e-05 2021-01-15T19:10:02.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252714+0000", 2021-01-15T19:10:02.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029657199999999999 2021-01-15T19:10:02.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253010+0000", 2021-01-15T19:10:02.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030464359 2021-01-15T19:10:02.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.283475+0000", 2021-01-15T19:10:02.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011930975999999999 2021-01-15T19:10:02.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295405+0000", 2021-01-15T19:10:02.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2021-01-15T19:10:02.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:02.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295488+0000", 2021-01-15T19:10:02.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074145999999999999 2021-01-15T19:10:02.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296229+0000", 2021-01-15T19:10:02.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999 2021-01-15T19:10:02.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.829 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.830 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1117 3.4s1 3:21d37027:::smithi00140323-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e114)", 2021-01-15T19:10:02.830 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:02.831 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.426002047000001, 2021-01-15T19:10:02.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003, 2021-01-15T19:10:02.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.831 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1117 2021-01-15T19:10:02.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.832 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:02.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:02.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3238000000000002e-05 2021-01-15T19:10:02.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252749+0000", 2021-01-15T19:10:02.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8820000000000001e-06 2021-01-15T19:10:02.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252757+0000", 2021-01-15T19:10:02.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6419e-05 2021-01-15T19:10:02.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252774+0000", 2021-01-15T19:10:02.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9879999999999998e-06 2021-01-15T19:10:02.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252781+0000", 2021-01-15T19:10:02.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051378399999999999 2021-01-15T19:10:02.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253294+0000", 2021-01-15T19:10:02.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8572000000000006e-05 2021-01-15T19:10:02.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:02.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253373+0000", 2021-01-15T19:10:02.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040965935000000002 2021-01-15T19:10:02.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294339+0000", 2021-01-15T19:10:02.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8033000000000001e-05 2021-01-15T19:10:02.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:02.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294437+0000", 2021-01-15T19:10:02.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018291189999999999 2021-01-15T19:10:02.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296266+0000", 2021-01-15T19:10:02.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0880999999999999e-05 2021-01-15T19:10:02.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296357+0000", 2021-01-15T19:10:02.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016417 2021-01-15T19:10:02.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296521+0000", 2021-01-15T19:10:02.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003 2021-01-15T19:10:02.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.846 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.847 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s0 3.18aae252 (undecoded) ondisk+write+peerstat_old+known_if_redirected e119)", 2021-01-15T19:10:02.847 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:02.847 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.399365714, 2021-01-15T19:10:02.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263, 2021-01-15T19:10:02.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:10:02.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.849 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:02.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:02.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5269000000000003e-05 2021-01-15T19:10:02.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279388+0000", 2021-01-15T19:10:02.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4849999999999998e-06 2021-01-15T19:10:02.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279397+0000", 2021-01-15T19:10:02.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.63e-05 2021-01-15T19:10:02.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279414+0000", 2021-01-15T19:10:02.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:02.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279422+0000", 2021-01-15T19:10:02.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5129e-05 2021-01-15T19:10:02.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279477+0000", 2021-01-15T19:10:02.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1559999999999997e-06 2021-01-15T19:10:02.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:02.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279481+0000", 2021-01-15T19:10:02.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.92165986799999999 2021-01-15T19:10:02.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201141+0000", 2021-01-15T19:10:02.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9066999999999999e-05 2021-01-15T19:10:02.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201160+0000", 2021-01-15T19:10:02.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263 2021-01-15T19:10:02.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.860 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.860 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4166.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e99)", 2021-01-15T19:10:02.861 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:02.861 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.122996491, 2021-01-15T19:10:02.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116, 2021-01-15T19:10:02.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4166", 2021-01-15T19:10:02.862 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4191424063", 2021-01-15T19:10:02.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4 2021-01-15T19:10:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:02.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:02.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7705999999999999e-05 2021-01-15T19:10:02.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555739+0000", 2021-01-15T19:10:02.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8659999999999995e-06 2021-01-15T19:10:02.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555748+0000", 2021-01-15T19:10:02.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2021-01-15T19:10:02.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555761+0000", 2021-01-15T19:10:02.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9859999999999994e-06 2021-01-15T19:10:02.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555769+0000", 2021-01-15T19:10:02.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034585600000000002 2021-01-15T19:10:02.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556115+0000", 2021-01-15T19:10:02.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:02.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:02.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556129+0000", 2021-01-15T19:10:02.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.65062051300000001 2021-01-15T19:10:02.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206749+0000", 2021-01-15T19:10:02.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014163399999999999 2021-01-15T19:10:02.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206891+0000", 2021-01-15T19:10:02.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116 2021-01-15T19:10:02.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.873 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.874 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.4s1 124/122 ECSubWrite(tid=344, reqid=client.4418.0:1180, at_version=124'63, trim_to=0'0, roll_forward_to=123'61))", 2021-01-15T19:10:02.874 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:02.874 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.446496128, 2021-01-15T19:10:02.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011987622999999999, 2021-01-15T19:10:02.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:02.875 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:02.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:02.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6241e-05 2021-01-15T19:10:02.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232258+0000", 2021-01-15T19:10:02.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:02.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232273+0000", 2021-01-15T19:10:02.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2021-01-15T19:10:02.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232281+0000", 2021-01-15T19:10:02.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3479999999999997e-06 2021-01-15T19:10:02.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232285+0000", 2021-01-15T19:10:02.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2518999999999998e-05 2021-01-15T19:10:02.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232318+0000", 2021-01-15T19:10:02.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:02.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232325+0000", 2021-01-15T19:10:02.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011821569000000001 2021-01-15T19:10:02.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.244147+0000", 2021-01-15T19:10:02.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3057999999999999e-05 2021-01-15T19:10:02.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.244200+0000", 2021-01-15T19:10:02.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011987622999999999 2021-01-15T19:10:02.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1187 3.12s0 3:4bacf681:::smithi00140323-17:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:02.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:02.887 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.4523231460000003, 2021-01-15T19:10:02.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999, 2021-01-15T19:10:02.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1187 2021-01-15T19:10:02.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.889 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:02.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:02.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8656999999999998e-05 2021-01-15T19:10:02.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226444+0000", 2021-01-15T19:10:02.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:02.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226452+0000", 2021-01-15T19:10:02.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3689000000000001e-05 2021-01-15T19:10:02.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226466+0000", 2021-01-15T19:10:02.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8540000000000002e-06 2021-01-15T19:10:02.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226472+0000", 2021-01-15T19:10:02.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2799999999999997e-05 2021-01-15T19:10:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226514+0000", 2021-01-15T19:10:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1725999999999999e-05 2021-01-15T19:10:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:02.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226526+0000", 2021-01-15T19:10:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066831169999999997 2021-01-15T19:10:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.233209+0000", 2021-01-15T19:10:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011647090000000001 2021-01-15T19:10:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234374+0000", 2021-01-15T19:10:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00061833799999999996 2021-01-15T19:10:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234992+0000", 2021-01-15T19:10:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001433157 2021-01-15T19:10:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.236425+0000", 2021-01-15T19:10:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044118559999999996 2021-01-15T19:10:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240837+0000", 2021-01-15T19:10:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152409 2021-01-15T19:10:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240990+0000", 2021-01-15T19:10:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999 2021-01-15T19:10:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.903 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1188 3.12s0 3:4bacf681:::smithi00140323-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:02.904 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.4521878859999999, 2021-01-15T19:10:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001, 2021-01-15T19:10:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1188 2021-01-15T19:10:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.906 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:02.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0703e-05 2021-01-15T19:10:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226561+0000", 2021-01-15T19:10:02.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5660000000000001e-06 2021-01-15T19:10:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226569+0000", 2021-01-15T19:10:02.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3385e-05 2021-01-15T19:10:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226582+0000", 2021-01-15T19:10:02.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226587+0000", 2021-01-15T19:10:02.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.409e-05 2021-01-15T19:10:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226632+0000", 2021-01-15T19:10:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1242e-05 2021-01-15T19:10:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226643+0000", 2021-01-15T19:10:02.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086436099999999995 2021-01-15T19:10:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235286+0000", 2021-01-15T19:10:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016845599999999999 2021-01-15T19:10:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:02.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235455+0000", 2021-01-15T19:10:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060319140000000002 2021-01-15T19:10:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241487+0000", 2021-01-15T19:10:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014580699999999999 2021-01-15T19:10:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241633+0000", 2021-01-15T19:10:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001527306 2021-01-15T19:10:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.243160+0000", 2021-01-15T19:10:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001 2021-01-15T19:10:02.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.921 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1192 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~557056] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.4230838009999998, 2021-01-15T19:10:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012911791000000001, 2021-01-15T19:10:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1192 2021-01-15T19:10:02.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042538899999999998 2021-01-15T19:10:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256050+0000", 2021-01-15T19:10:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1664999999999999e-05 2021-01-15T19:10:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256061+0000", 2021-01-15T19:10:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0514e-05 2021-01-15T19:10:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256082+0000", 2021-01-15T19:10:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0865e-05 2021-01-15T19:10:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256093+0000", 2021-01-15T19:10:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2021-01-15T19:10:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256160+0000", 2021-01-15T19:10:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013985700000000001 2021-01-15T19:10:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256300+0000", 2021-01-15T19:10:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001360053 2021-01-15T19:10:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.257660+0000", 2021-01-15T19:10:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010289375999999999 2021-01-15T19:10:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267950+0000", 2021-01-15T19:10:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051317300000000001 2021-01-15T19:10:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268463+0000", 2021-01-15T19:10:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3504999999999995e-05 2021-01-15T19:10:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268536+0000", 2021-01-15T19:10:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012911791000000001 2021-01-15T19:10:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.936 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.937 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1193 3.12s0 3:4e4266df:::smithi00140323-37:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.4225665340000004, 2021-01-15T19:10:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013858986, 2021-01-15T19:10:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1193 2021-01-15T19:10:02.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7895999999999997e-05 2021-01-15T19:10:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256190+0000", 2021-01-15T19:10:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5399999999999998e-06 2021-01-15T19:10:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256197+0000", 2021-01-15T19:10:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.252e-05 2021-01-15T19:10:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256210+0000", 2021-01-15T19:10:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1340000000000001e-06 2021-01-15T19:10:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256216+0000", 2021-01-15T19:10:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010534175 2021-01-15T19:10:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.266750+0000", 2021-01-15T19:10:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109787 2021-01-15T19:10:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.266860+0000", 2021-01-15T19:10:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00057329899999999999 2021-01-15T19:10:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:02.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267433+0000", 2021-01-15T19:10:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076203700000000002 2021-01-15T19:10:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:02.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268195+0000", 2021-01-15T19:10:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001719424 2021-01-15T19:10:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:02.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.269914+0000", 2021-01-15T19:10:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6174000000000005e-05 2021-01-15T19:10:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270001+0000", 2021-01-15T19:10:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013858986 2021-01-15T19:10:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1194 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:02.953 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.4224449269999999, 2021-01-15T19:10:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019, 2021-01-15T19:10:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:02.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1194 2021-01-15T19:10:02.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:02.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8844999999999998e-05 2021-01-15T19:10:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256302+0000", 2021-01-15T19:10:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9030000000000004e-06 2021-01-15T19:10:02.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256310+0000", 2021-01-15T19:10:02.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4749999999999999e-05 2021-01-15T19:10:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256325+0000", 2021-01-15T19:10:02.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2370000000000003e-06 2021-01-15T19:10:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256332+0000", 2021-01-15T19:10:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011409671999999999 2021-01-15T19:10:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267742+0000", 2021-01-15T19:10:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017902200000000001 2021-01-15T19:10:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267921+0000", 2021-01-15T19:10:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027501040000000002 2021-01-15T19:10:02.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270671+0000", 2021-01-15T19:10:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020387700000000001 2021-01-15T19:10:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270875+0000", 2021-01-15T19:10:02.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012356089999999999 2021-01-15T19:10:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:02.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.272110+0000", 2021-01-15T19:10:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019 2021-01-15T19:10:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:02.970 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:02.970 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:02.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:02.972 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:02.971+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:02.973 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:03.073 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:03.072+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:03.148 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:03.150 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:03.159 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:03.162 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:03.173 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:03.173+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:03.182 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:03.183 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:03.203 INFO:tasks.rados.rados.0.smithi001.stdout:533: done (4 left) 2021-01-15T19:10:03.203 INFO:tasks.rados.rados.0.smithi001.stdout:534: read oid 13 snap 55 2021-01-15T19:10:03.204 INFO:tasks.rados.rados.0.smithi001.stdout:534: expect (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:10:03.204 INFO:tasks.rados.rados.0.smithi001.stdout:535: rollback oid 48 current snap is 58 2021-01-15T19:10:03.204 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 48 to 56 2021-01-15T19:10:03.209 INFO:tasks.rados.rados.0.smithi001.stdout:529: finishing write tid 1 to smithi00140323-8 2021-01-15T19:10:03.210 INFO:tasks.rados.rados.0.smithi001.stdout:529: finishing write tid 2 to smithi00140323-8 2021-01-15T19:10:03.210 INFO:tasks.rados.rados.0.smithi001.stdout:529: finishing write tid 3 to smithi00140323-8 2021-01-15T19:10:03.210 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 149 (ObjNum 188 snap 58 seq_num 188) dirty exists 2021-01-15T19:10:03.211 INFO:tasks.rados.rados.0.smithi001.stdout:529: left oid 8 (ObjNum 188 snap 58 seq_num 188) 2021-01-15T19:10:03.211 INFO:tasks.rados.rados.0.smithi001.stdout:530: finishing write tid 1 to smithi00140323-10 2021-01-15T19:10:03.211 INFO:tasks.rados.rados.0.smithi001.stdout:530: finishing write tid 2 to smithi00140323-10 2021-01-15T19:10:03.211 INFO:tasks.rados.rados.0.smithi001.stdout:530: finishing write tid 3 to smithi00140323-10 2021-01-15T19:10:03.211 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 79 (ObjNum 189 snap 58 seq_num 189) dirty exists 2021-01-15T19:10:03.212 INFO:tasks.rados.rados.0.smithi001.stdout:530: left oid 10 (ObjNum 189 snap 58 seq_num 189) 2021-01-15T19:10:03.212 INFO:tasks.rados.rados.0.smithi001.stdout:531: expect (ObjNum 173 snap 52 seq_num 173) 2021-01-15T19:10:03.218 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:03.265 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:03.274 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:03.274+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:03.299 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:03.317 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:03.317 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:03.320 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:03.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:03.333 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:03.333 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:03.333 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:03.342 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:03.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:10:03.367 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:03.369 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.370 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:03.384 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:03.382+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:03.477 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:03.475+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:03.482 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:03.483 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:03.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:03.550 INFO:tasks.rados.rados.0.smithi001.stdout:535: finishing rollback tid 0 to smithi00140323-48 2021-01-15T19:10:03.550 INFO:tasks.rados.rados.0.smithi001.stdout:535: finishing rollback tid 1 to smithi00140323-48 2021-01-15T19:10:03.550 INFO:tasks.rados.rados.0.smithi001.stdout:535: finishing rollback tid 2 to smithi00140323-48 2021-01-15T19:10:03.551 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 48 v 148 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:10:03.551 INFO:tasks.rados.rados.0.smithi001.stdout:529: done (5 left) 2021-01-15T19:10:03.551 INFO:tasks.rados.rados.0.smithi001.stdout:530: done (4 left) 2021-01-15T19:10:03.551 INFO:tasks.rados.rados.0.smithi001.stdout:531: done (3 left) 2021-01-15T19:10:03.551 INFO:tasks.rados.rados.0.smithi001.stdout:532: done (2 left) 2021-01-15T19:10:03.552 INFO:tasks.rados.rados.0.smithi001.stdout:535: done (1 left) 2021-01-15T19:10:03.552 INFO:tasks.rados.rados.0.smithi001.stdout:536: rollback oid 12 current snap is 58 2021-01-15T19:10:03.552 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 12 to 54 2021-01-15T19:10:03.552 INFO:tasks.rados.rados.0.smithi001.stdout:534: expect (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:10:03.578 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:03.576+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:03.669 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:03.670 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:03.678 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:03.677+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:03.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:03.779 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:03.778+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:03.853 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:03.854 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:03.866 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:03.880 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:03.880+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:03.957 INFO:tasks.rados.rados.0.smithi001.stdout:536: finishing rollback tid 0 to smithi00140323-12 2021-01-15T19:10:03.957 INFO:tasks.rados.rados.0.smithi001.stdout:536: finishing rollback tid 1 to smithi00140323-12 2021-01-15T19:10:03.958 INFO:tasks.rados.rados.0.smithi001.stdout:536: finishing rollback tid 2 to smithi00140323-12 2021-01-15T19:10:03.958 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 81 (ObjNum 127 snap 30 seq_num 127) dirty exists 2021-01-15T19:10:03.958 INFO:tasks.rados.rados.0.smithi001.stdout:534: done (1 left) 2021-01-15T19:10:03.958 INFO:tasks.rados.rados.0.smithi001.stdout:536: done (0 left) 2021-01-15T19:10:03.959 INFO:tasks.rados.rados.0.smithi001.stdout:537: snap_create 2021-01-15T19:10:03.981 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:03.980+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:04.038 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.039 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:04.089 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:04.088+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:04.174 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.175 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.184 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:04.184 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:04.182+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:04.284 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:04.282+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:04.311 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.311 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.321 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:04.331 INFO:tasks.rados.rados.0.smithi001.stdout:537: done (0 left) 2021-01-15T19:10:04.331 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 5 current snap is 59 2021-01-15T19:10:04.331 INFO:tasks.rados.rados.0.smithi001.stdout:538: seq_num 190 ranges {688128=786432} 2021-01-15T19:10:04.333 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:10:04.339 INFO:tasks.rados.rados.0.smithi001.stdout:538: writing smithi00140323-5 from 688128 to 1474560 tid 1 2021-01-15T19:10:04.340 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 1 current snap is 59 2021-01-15T19:10:04.340 INFO:tasks.rados.rados.0.smithi001.stdout:539: seq_num 191 ranges {0=802816} 2021-01-15T19:10:04.344 INFO:tasks.rados.rados.0.smithi001.stdout:539: writing smithi00140323-1 from 0 to 802816 tid 1 2021-01-15T19:10:04.344 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 59 2021-01-15T19:10:04.345 INFO:tasks.rados.rados.0.smithi001.stdout:540: seq_num 192 ranges {6692864=417792,7110656=401408} 2021-01-15T19:10:04.384 INFO:tasks.rados.rados.0.smithi001.stdout:540: writing smithi00140323-13 from 6692864 to 7110656 tid 1 2021-01-15T19:10:04.384 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:04.383+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:04.386 INFO:tasks.rados.rados.0.smithi001.stdout:540: writing smithi00140323-13 from 7110656 to 7512064 tid 2 2021-01-15T19:10:04.386 INFO:tasks.rados.rados.0.smithi001.stdout:541: delete oid 33 current snap is 59 2021-01-15T19:10:04.386 INFO:tasks.rados.rados.0.smithi001.stdout:538: finishing write tid 1 to smithi00140323-5 2021-01-15T19:10:04.387 INFO:tasks.rados.rados.0.smithi001.stdout:538: finishing write tid 2 to smithi00140323-5 2021-01-15T19:10:04.387 INFO:tasks.rados.rados.0.smithi001.stdout:539: finishing write tid 1 to smithi00140323-1 2021-01-15T19:10:04.387 INFO:tasks.rados.rados.0.smithi001.stdout:538: finishing write tid 3 to smithi00140323-5 2021-01-15T19:10:04.387 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 5 v 143 (ObjNum 190 snap 59 seq_num 190) dirty exists 2021-01-15T19:10:04.388 INFO:tasks.rados.rados.0.smithi001.stdout:538: left oid 5 (ObjNum 190 snap 59 seq_num 190) 2021-01-15T19:10:04.388 INFO:tasks.rados.rados.0.smithi001.stdout:539: finishing write tid 2 to smithi00140323-1 2021-01-15T19:10:04.388 INFO:tasks.rados.rados.0.smithi001.stdout:539: finishing write tid 3 to smithi00140323-1 2021-01-15T19:10:04.388 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 146 (ObjNum 191 snap 59 seq_num 191) dirty exists 2021-01-15T19:10:04.388 INFO:tasks.rados.rados.0.smithi001.stdout:539: left oid 1 (ObjNum 191 snap 59 seq_num 191) 2021-01-15T19:10:04.397 INFO:tasks.rados.rados.0.smithi001.stdout:538: done (3 left) 2021-01-15T19:10:04.397 INFO:tasks.rados.rados.0.smithi001.stdout:539: done (2 left) 2021-01-15T19:10:04.397 INFO:tasks.rados.rados.0.smithi001.stdout:541: done (1 left) 2021-01-15T19:10:04.398 INFO:tasks.rados.rados.0.smithi001.stdout:542: read oid 7 snap -1 2021-01-15T19:10:04.398 INFO:tasks.rados.rados.0.smithi001.stdout:542: expect (ObjNum 183 snap 58 seq_num 183) 2021-01-15T19:10:04.398 INFO:tasks.rados.rados.0.smithi001.stdout:543: snap_remove snap 46 2021-01-15T19:10:04.455 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.456 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.465 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:04.506 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:04.505+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:04.586 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:04.585+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.639 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:04.686 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:04.685+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:04.763 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.764 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.773 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:04.787 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:04.787+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:04.889 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:04.888+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:04.898 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:04.898 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:04.908 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:04.990 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:04.989+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:05.073 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:05.073 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:05.083 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:05.090 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:05.089+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:05.191 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:05.190+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:05.207 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:05.208 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:05.217 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:05.292 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:05.291+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:05.334 INFO:tasks.rados.rados.0.smithi001.stdout:543: done (2 left) 2021-01-15T19:10:05.334 INFO:tasks.rados.rados.0.smithi001.stdout:544: rollback oid 21 current snap is 59 2021-01-15T19:10:05.334 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 21 to 45 2021-01-15T19:10:05.335 INFO:tasks.rados.rados.0.smithi001.stdout:540: finishing write tid 1 to smithi00140323-13 2021-01-15T19:10:05.335 INFO:tasks.rados.rados.0.smithi001.stdout:540: finishing write tid 2 to smithi00140323-13 2021-01-15T19:10:05.335 INFO:tasks.rados.rados.0.smithi001.stdout:540: finishing write tid 3 to smithi00140323-13 2021-01-15T19:10:05.335 INFO:tasks.rados.rados.0.smithi001.stdout:540: finishing write tid 4 to smithi00140323-13 2021-01-15T19:10:05.335 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 236 (ObjNum 192 snap 59 seq_num 192) dirty exists 2021-01-15T19:10:05.336 INFO:tasks.rados.rados.0.smithi001.stdout:540: left oid 13 (ObjNum 192 snap 59 seq_num 192) 2021-01-15T19:10:05.336 INFO:tasks.rados.rados.0.smithi001.stdout:542: expect (ObjNum 183 snap 58 seq_num 183) 2021-01-15T19:10:05.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:05.392 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:05.392 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:05.393 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:05.391+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:05.402 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:05.494 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:05.493+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:05.537 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:05.558 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:05.569 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:05.569 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:05.577 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:05.579 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:05.594 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:05.593+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:05.597 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:05.618 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:05.639 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:05.659 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:05.680 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:05.695 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:05.694+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:05.745 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:05.753 INFO:tasks.rados.rados.0.smithi001.stdout:544: finishing rollback tid 0 to smithi00140323-21 2021-01-15T19:10:05.753 INFO:tasks.rados.rados.0.smithi001.stdout:540: done (2 left) 2021-01-15T19:10:05.753 INFO:tasks.rados.rados.0.smithi001.stdout:542: done (1 left) 2021-01-15T19:10:05.754 INFO:tasks.rados.rados.0.smithi001.stdout:545: snap_create 2021-01-15T19:10:05.754 INFO:tasks.rados.rados.0.smithi001.stdout:544: finishing rollback tid 1 to smithi00140323-21 2021-01-15T19:10:05.754 INFO:tasks.rados.rados.0.smithi001.stdout:544: finishing rollback tid 2 to smithi00140323-21 2021-01-15T19:10:05.754 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 102 (ObjNum 130 snap 31 seq_num 130) dirty exists 2021-01-15T19:10:05.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:05.796 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:05.795+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:05.897 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:05.895+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:05.926 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:05.927 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:05.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:05.997 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:05.997+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:06.069 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:06.069 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:06.069 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:06.070 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:06.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:06.122 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:06.121+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:06.199 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:06.197+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:06.210 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:06.211 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:06.211 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:06.211 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:06.211 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:06.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:06.304 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:06.303+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:06.338 INFO:tasks.rados.rados.0.smithi001.stdout:544: done (1 left) 2021-01-15T19:10:06.339 INFO:tasks.rados.rados.0.smithi001.stdout:545: done (0 left) 2021-01-15T19:10:06.339 INFO:tasks.rados.rados.0.smithi001.stdout:546: read oid 16 snap -1 2021-01-15T19:10:06.339 INFO:tasks.rados.rados.0.smithi001.stdout:546: expect deleted 2021-01-15T19:10:06.339 INFO:tasks.rados.rados.0.smithi001.stdout:547: copy_from oid 26 from oid 35 current snap is 60 2021-01-15T19:10:06.340 INFO:tasks.rados.rados.0.smithi001.stdout:548: delete oid 29 current snap is 60 2021-01-15T19:10:06.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:10:06.347 INFO:tasks.rados.rados.0.smithi001.stdout:546: done (2 left) 2021-01-15T19:10:06.347 INFO:tasks.rados.rados.0.smithi001.stdout:548: done (1 left) 2021-01-15T19:10:06.347 INFO:tasks.rados.rados.0.smithi001.stdout:549: rmattr oid 33 current snap is 60 2021-01-15T19:10:06.348 INFO:tasks.rados.rados.0.smithi001.stdout:549: done (1 left) 2021-01-15T19:10:06.348 INFO:tasks.rados.rados.0.smithi001.stdout:550: delete oid 25 current snap is 60 2021-01-15T19:10:06.357 INFO:tasks.rados.rados.0.smithi001.stdout:547: finishing copy_from to smithi00140323-26 2021-01-15T19:10:06.357 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 26 v 120 (ObjNum 157 snap 41 seq_num 157) dirty exists 2021-01-15T19:10:06.358 INFO:tasks.rados.rados.0.smithi001.stdout:550: done (1 left) 2021-01-15T19:10:06.358 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 10 current snap is 60 2021-01-15T19:10:06.358 INFO:tasks.rados.rados.0.smithi001.stdout:551: seq_num 196 ranges {3956736=491520,4448256=401408} 2021-01-15T19:10:06.383 INFO:tasks.rados.rados.0.smithi001.stdout:551: writing smithi00140323-10 from 3956736 to 4448256 tid 1 2021-01-15T19:10:06.385 INFO:tasks.rados.rados.0.smithi001.stdout:551: writing smithi00140323-10 from 4448256 to 4849664 tid 2 2021-01-15T19:10:06.385 INFO:tasks.rados.rados.0.smithi001.stdout:552: snap_create 2021-01-15T19:10:06.386 INFO:tasks.rados.rados.0.smithi001.stdout:547: finishing copy_from racing read to smithi00140323-26 2021-01-15T19:10:06.391 INFO:tasks.rados.rados.0.smithi001.stdout:551: finishing write tid 1 to smithi00140323-10 2021-01-15T19:10:06.392 INFO:tasks.rados.rados.0.smithi001.stdout:551: finishing write tid 2 to smithi00140323-10 2021-01-15T19:10:06.393 INFO:tasks.rados.rados.0.smithi001.stdout:551: finishing write tid 3 to smithi00140323-10 2021-01-15T19:10:06.395 INFO:tasks.rados.rados.0.smithi001.stdout:551: finishing write tid 4 to smithi00140323-10 2021-01-15T19:10:06.395 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 10 v 87 (ObjNum 196 snap 60 seq_num 196) dirty exists 2021-01-15T19:10:06.396 INFO:tasks.rados.rados.0.smithi001.stdout:551: left oid 10 (ObjNum 196 snap 60 seq_num 196) 2021-01-15T19:10:06.401 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:06.400+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:06.412 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:06.413 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:06.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:06.413 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:06.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.413 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:06.414 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:06.414 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.157300608, 2021-01-15T19:10:06.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:06.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.415 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:06.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.416 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:06.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:06.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:06.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:06.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:06.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:06.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:06.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:06.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:06.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:06.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:06.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:06.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:06.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:06.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:06.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:06.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:06.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:06.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:06.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:06.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:06.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:06.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:06.428 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.428 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.429 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:06.429 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:06.430 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.157135495000006, 2021-01-15T19:10:06.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:06.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.431 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:06.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.431 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:06.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:06.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:06.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:06.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:06.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:06.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:06.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:06.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:06.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:06.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:06.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:06.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:06.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:06.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:06.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:06.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:06.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:06.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:06.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:06.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:06.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:06.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:06.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:06.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:06.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:06.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.446 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:06.446 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:06.446 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.149855446000004, 2021-01-15T19:10:06.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:06.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:06.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.448 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:06.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:06.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:06.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:06.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:06.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:06.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:06.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:06.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:06.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:06.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:06.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:06.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:06.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:06.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:06.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:06.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:06.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:06.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:06.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:06.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:06.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:06.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:06.460 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.461 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.461 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:06.462 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:06.462 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.149728812999996, 2021-01-15T19:10:06.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:06.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:06.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:06.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:06.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:06.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:06.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:06.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:06.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:06.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:06.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:06.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:06.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:06.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:06.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:06.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:06.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:06.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:06.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:06.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:06.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:06.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:06.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:06.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:06.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:06.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:06.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:06.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.477 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.477 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.478 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:06.478 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:06.478 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.066991848000001, 2021-01-15T19:10:06.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:06.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.479 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:06.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.480 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:06.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:06.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:06.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:06.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:06.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:06.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:06.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:06.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:06.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:06.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:06.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:06.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:06.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:06.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:06.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:06.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:06.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:06.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:06.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:06.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:06.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:06.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:06.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.492 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:06.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:06.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.066842233999999, 2021-01-15T19:10:06.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:06.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:06.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.496 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:06.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:06.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:06.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:06.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:06.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:06.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:06.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:06.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:06.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:06.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:06.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:06.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:06.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:06.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:06.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:06.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:06.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:06.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:06.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:06.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:06.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:06.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:06.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:06.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:06.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:06.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.511 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:06.511 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:06.511 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.046544552, 2021-01-15T19:10:06.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:06.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:06.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.513 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:06.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:06.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:06.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:06.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:06.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:06.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:06.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:06.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:06.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:06.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:06.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:06.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:06.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:06.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:06.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:06.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:06.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:06.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:06.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:06.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:06.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:06.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:06.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:06.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:06.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.527 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.528 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:06.528 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:06.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.043487900999999, 2021-01-15T19:10:06.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:06.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.529 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:06.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.530 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:06.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:06.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:06.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:06.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:06.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:06.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:06.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:06.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:06.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:06.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:06.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:06.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:06.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:06.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:06.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:06.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:06.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:06.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:06.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:06.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.040474639000003, 2021-01-15T19:10:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:06.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.562 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:06.563 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.037727027999999, 2021-01-15T19:10:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.565 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:06.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:06.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:06.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:06.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:06.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.037293206000001, 2021-01-15T19:10:06.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.582 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.597 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:06.598 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 54.037169046999999, 2021-01-15T19:10:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.599 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.600 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:06.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:06.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:06.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:06.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:06.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:06.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:06.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:06.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:06.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:06.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.932492512000003, 2021-01-15T19:10:06.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.616 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:06.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:06.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:06.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:06.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:06.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:06.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:06.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:06.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:06.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:06.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:06.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:06.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:06.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:06.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:06.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:06.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:06.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:06.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:06.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:06.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:06.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:06.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:06.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:06.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:06.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:06.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:06.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:06.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:06.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:06.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:06.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:06.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:06.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:06.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:06.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:06.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:06.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:06.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:06.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:06.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.636 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.636 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:06.637 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:06.637 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.052271433000001, 2021-01-15T19:10:06.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:06.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.638 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:06.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:06.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:06.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:06.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:06.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:06.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:06.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:06.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:06.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:06.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:06.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:06.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:06.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:06.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:06.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:06.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:06.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.647 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:06.648 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:06.648 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.022553275999996, 2021-01-15T19:10:06.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:06.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:06.649 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:06.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:06.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:06.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:06.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:06.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:06.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:06.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:06.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:06.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:06.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:06.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:06.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:06.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:06.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:06.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.658 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.659 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:06.659 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:06.659 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.018467108999999, 2021-01-15T19:10:06.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:06.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:06.660 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:06.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:06.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:06.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:06.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:06.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:06.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:06.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:06.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:06.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:06.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:06.668 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.018331357999998, 2021-01-15T19:10:06.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.679 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=28'16, committed=1, applied=1))", 2021-01-15T19:10:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:10:06.680 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.018104508999997, 2021-01-15T19:10:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022959151000000001, 2021-01-15T19:10:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:10:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392458+0000", 2021-01-15T19:10:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6398999999999997e-05 2021-01-15T19:10:06.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392544+0000", 2021-01-15T19:10:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1029999999999999e-05 2021-01-15T19:10:06.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392555+0000", 2021-01-15T19:10:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4134999999999999e-05 2021-01-15T19:10:06.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392569+0000", 2021-01-15T19:10:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.874e-06 2021-01-15T19:10:06.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392575+0000", 2021-01-15T19:10:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022133907000000001 2021-01-15T19:10:06.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414709+0000", 2021-01-15T19:10:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00070780599999999995 2021-01-15T19:10:06.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415417+0000", 2021-01-15T19:10:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022959151000000001 2021-01-15T19:10:06.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.690 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.179832375, 2021-01-15T19:10:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:06.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:06.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:06.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:06.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:06.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:06.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:06.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1285 3.cs0 3:3f312ec7:::smithi00140323-7:64 [read 0~5472256,checksum] snapc 0=[] ondisk+read+known_if_redirected e134)", 2021-01-15T19:10:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:02.426227+0000", 2021-01-15T19:10:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.9843356750000001, 2021-01-15T19:10:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023059988, 2021-01-15T19:10:06.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:06.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1285 2021-01-15T19:10:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:06.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.426227+0000", 2021-01-15T19:10:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.426227+0000", 2021-01-15T19:10:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7488999999999998e-05 2021-01-15T19:10:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:06.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.426284+0000", 2021-01-15T19:10:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6810000000000001e-06 2021-01-15T19:10:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:06.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.426291+0000", 2021-01-15T19:10:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4320999999999999e-05 2021-01-15T19:10:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:06.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.426325+0000", 2021-01-15T19:10:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0969999999999996e-06 2021-01-15T19:10:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:06.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.426330+0000", 2021-01-15T19:10:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0045831500000000002 2021-01-15T19:10:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:06.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.430913+0000", 2021-01-15T19:10:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020178700000000001 2021-01-15T19:10:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:06.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.431115+0000", 2021-01-15T19:10:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018171462999999999 2021-01-15T19:10:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:06.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:02.449287+0000", 2021-01-15T19:10:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023059988 2021-01-15T19:10:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:06.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:06.717 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:06.718 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:10:06.719 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:06.717+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:06.819 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:06.818+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:06.890 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:06.891 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:06.891 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:06.891 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:06.901 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:10:06.919 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:06.918+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:07.021 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:07.019+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:07.072 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:07.072 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:07.072 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:07.073 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:07.073 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:07.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:10:07.122 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:07.121+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:07.223 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:07.222+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:07.257 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:07.257 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:07.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:07.258 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:07.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.258 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:10:07.259 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:07.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.73853816600001, 2021-01-15T19:10:07.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:10:07.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:10:07.260 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:10:07.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:10:07.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.261 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:07.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:07.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:10:07.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:10:07.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:10:07.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:10:07.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:10:07.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:10:07.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:10:07.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:10:07.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:10:07.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:10:07.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:10:07.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:07.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:10:07.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:10:07.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:10:07.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:10:07.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:10:07.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:10:07.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.271 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.272 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:828 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:07.272 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:10:07.273 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.072897619999999, 2021-01-15T19:10:07.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011908975, 2021-01-15T19:10:07.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.274 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 828 2021-01-15T19:10:07.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.275 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:10:07.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182495+0000", 2021-01-15T19:10:07.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9705000000000001e-05 2021-01-15T19:10:07.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182525+0000", 2021-01-15T19:10:07.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6079999999999998e-06 2021-01-15T19:10:07.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182529+0000", 2021-01-15T19:10:07.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9899999999999997e-06 2021-01-15T19:10:07.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182537+0000", 2021-01-15T19:10:07.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1290000000000004e-06 2021-01-15T19:10:07.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182542+0000", 2021-01-15T19:10:07.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2103999999999999e-05 2021-01-15T19:10:07.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182615+0000", 2021-01-15T19:10:07.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0329999999999992e-06 2021-01-15T19:10:07.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:07.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:07.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040604489999999998 2021-01-15T19:10:07.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.186683+0000", 2021-01-15T19:10:07.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116184 2021-01-15T19:10:07.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.186799+0000", 2021-01-15T19:10:07.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0076047729999999996 2021-01-15T19:10:07.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.194404+0000", 2021-01-15T19:10:07.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011908975 2021-01-15T19:10:07.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:829 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.072823845999999, 2021-01-15T19:10:07.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001, 2021-01-15T19:10:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 829 2021-01-15T19:10:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4360000000000001e-05 2021-01-15T19:10:07.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0169999999999999e-06 2021-01-15T19:10:07.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182630+0000", 2021-01-15T19:10:07.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4198e-05 2021-01-15T19:10:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182644+0000", 2021-01-15T19:10:07.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1253999999999999e-05 2021-01-15T19:10:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182656+0000", 2021-01-15T19:10:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9756999999999999e-05 2021-01-15T19:10:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182695+0000", 2021-01-15T19:10:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2819999999999996e-06 2021-01-15T19:10:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182704+0000", 2021-01-15T19:10:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048051730000000003 2021-01-15T19:10:07.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187509+0000", 2021-01-15T19:10:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114002 2021-01-15T19:10:07.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187623+0000", 2021-01-15T19:10:07.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012123558 2021-01-15T19:10:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.199746+0000", 2021-01-15T19:10:07.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001 2021-01-15T19:10:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=315, reqid=client.4418.0:833, at_version=80'84, trim_to=0'0, roll_forward_to=78'82))", 2021-01-15T19:10:07.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.068261878999998, 2021-01-15T19:10:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001, 2021-01-15T19:10:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.305 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:07.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:07.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6477000000000002e-05 2021-01-15T19:10:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187187+0000", 2021-01-15T19:10:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4749e-05 2021-01-15T19:10:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187212+0000", 2021-01-15T19:10:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5977e-05 2021-01-15T19:10:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187228+0000", 2021-01-15T19:10:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0770000000000008e-06 2021-01-15T19:10:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187236+0000", 2021-01-15T19:10:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014010800000000001 2021-01-15T19:10:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187376+0000", 2021-01-15T19:10:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8624999999999999e-05 2021-01-15T19:10:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187395+0000", 2021-01-15T19:10:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013611689 2021-01-15T19:10:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201006+0000", 2021-01-15T19:10:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7980999999999999e-05 2021-01-15T19:10:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201064+0000", 2021-01-15T19:10:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001 2021-01-15T19:10:07.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.317 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:971 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.046118485000001, 2021-01-15T19:10:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002, 2021-01-15T19:10:07.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.319 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 971 2021-01-15T19:10:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2442999999999998e-05 2021-01-15T19:10:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209337+0000", 2021-01-15T19:10:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0950000000000002e-06 2021-01-15T19:10:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209346+0000", 2021-01-15T19:10:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9199999999999999e-05 2021-01-15T19:10:07.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209365+0000", 2021-01-15T19:10:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5750000000000007e-06 2021-01-15T19:10:07.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209374+0000", 2021-01-15T19:10:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074262999999999998 2021-01-15T19:10:07.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210117+0000", 2021-01-15T19:10:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022372900000000001 2021-01-15T19:10:07.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210341+0000", 2021-01-15T19:10:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016740338 2021-01-15T19:10:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227081+0000", 2021-01-15T19:10:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002 2021-01-15T19:10:07.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:972 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.045971729000001, 2021-01-15T19:10:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999, 2021-01-15T19:10:07.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 972 2021-01-15T19:10:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5875000000000003e-05 2021-01-15T19:10:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209477+0000", 2021-01-15T19:10:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7499999999999992e-06 2021-01-15T19:10:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209486+0000", 2021-01-15T19:10:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4825e-05 2021-01-15T19:10:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209500+0000", 2021-01-15T19:10:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209508+0000", 2021-01-15T19:10:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012352109999999999 2021-01-15T19:10:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210743+0000", 2021-01-15T19:10:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012467700000000001 2021-01-15T19:10:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210867+0000", 2021-01-15T19:10:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018560825 2021-01-15T19:10:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.229428+0000", 2021-01-15T19:10:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999 2021-01-15T19:10:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:973 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:07.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.045840396999999, 2021-01-15T19:10:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001, 2021-01-15T19:10:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 973 2021-01-15T19:10:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:07.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:07.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7021999999999997e-05 2021-01-15T19:10:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209609+0000", 2021-01-15T19:10:07.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3140000000000004e-06 2021-01-15T19:10:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209618+0000", 2021-01-15T19:10:07.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4259e-05 2021-01-15T19:10:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209632+0000", 2021-01-15T19:10:07.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2021-01-15T19:10:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209639+0000", 2021-01-15T19:10:07.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039100000000000003 2021-01-15T19:10:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213549+0000", 2021-01-15T19:10:07.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124948 2021-01-15T19:10:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213674+0000", 2021-01-15T19:10:07.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018190644999999998 2021-01-15T19:10:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.231864+0000", 2021-01-15T19:10:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001 2021-01-15T19:10:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1093 3.10s0 3:095f0b32:::smithi00140323-47:head [rollback 50] snapc 50=[50,4b,44,41,26] ondisk+write+known_if_redirected e109)", 2021-01-15T19:10:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.019166263999999, 2021-01-15T19:10:07.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242, 2021-01-15T19:10:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1093 2021-01-15T19:10:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3699999999999998e-05 2021-01-15T19:10:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236270+0000", 2021-01-15T19:10:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4700000000000008e-06 2021-01-15T19:10:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236280+0000", 2021-01-15T19:10:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6716999999999999e-05 2021-01-15T19:10:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236296+0000", 2021-01-15T19:10:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5490000000000006e-06 2021-01-15T19:10:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236305+0000", 2021-01-15T19:10:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4991999999999998e-05 2021-01-15T19:10:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236360+0000", 2021-01-15T19:10:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3660000000000003e-06 2021-01-15T19:10:07.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236367+0000", 2021-01-15T19:10:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005477549 2021-01-15T19:10:07.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241845+0000", 2021-01-15T19:10:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137458 2021-01-15T19:10:07.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241982+0000", 2021-01-15T19:10:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097224410000000001 2021-01-15T19:10:07.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.251705+0000", 2021-01-15T19:10:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242 2021-01-15T19:10:07.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 109/95 ECSubWrite(tid=263, reqid=client.4418.0:1090, at_version=109'60, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 30.015350626, 2021-01-15T19:10:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999, 2021-01-15T19:10:07.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8936e-05 2021-01-15T19:10:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240091+0000", 2021-01-15T19:10:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1778000000000001e-05 2021-01-15T19:10:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240113+0000", 2021-01-15T19:10:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6714999999999998e-05 2021-01-15T19:10:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240129+0000", 2021-01-15T19:10:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240137+0000", 2021-01-15T19:10:07.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9104999999999998e-05 2021-01-15T19:10:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240186+0000", 2021-01-15T19:10:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240194+0000", 2021-01-15T19:10:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017479262999999998 2021-01-15T19:10:07.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257674+0000", 2021-01-15T19:10:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9951999999999999e-05 2021-01-15T19:10:07.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257724+0000", 2021-01-15T19:10:07.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999 2021-01-15T19:10:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 110/95 ECSubWrite(tid=269, reqid=client.4418.0:1098, at_version=110'64, trim_to=0'0, roll_forward_to=109'62))", 2021-01-15T19:10:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.061289616, 2021-01-15T19:10:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999, 2021-01-15T19:10:07.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5725e-05 2021-01-15T19:10:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194149+0000", 2021-01-15T19:10:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3603000000000003e-05 2021-01-15T19:10:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194182+0000", 2021-01-15T19:10:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0721e-05 2021-01-15T19:10:07.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194203+0000", 2021-01-15T19:10:07.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3929999999999997e-06 2021-01-15T19:10:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194212+0000", 2021-01-15T19:10:07.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0593000000000006e-05 2021-01-15T19:10:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194293+0000", 2021-01-15T19:10:07.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3596e-05 2021-01-15T19:10:07.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194307+0000", 2021-01-15T19:10:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017420983000000001 2021-01-15T19:10:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211728+0000", 2021-01-15T19:10:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6654999999999999e-05 2021-01-15T19:10:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211784+0000", 2021-01-15T19:10:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999 2021-01-15T19:10:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1116 3.4s1 3:21d37027:::smithi00140323-27:head [copy-from ver 54] snapc 57=[57,56,53,50,4b,44,41,26] ondisk+write+known_if_redirected e114)", 2021-01-15T19:10:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.002851014000001, 2021-01-15T19:10:07.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999, 2021-01-15T19:10:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1116 2021-01-15T19:10:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3304000000000002e-05 2021-01-15T19:10:07.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252605+0000", 2021-01-15T19:10:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0723000000000001e-05 2021-01-15T19:10:07.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252616+0000", 2021-01-15T19:10:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0720000000000002e-05 2021-01-15T19:10:07.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252636+0000", 2021-01-15T19:10:07.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1799e-05 2021-01-15T19:10:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252648+0000", 2021-01-15T19:10:07.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5387000000000003e-05 2021-01-15T19:10:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252714+0000", 2021-01-15T19:10:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029657199999999999 2021-01-15T19:10:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253010+0000", 2021-01-15T19:10:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030464359 2021-01-15T19:10:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.283475+0000", 2021-01-15T19:10:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011930975999999999 2021-01-15T19:10:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295405+0000", 2021-01-15T19:10:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2021-01-15T19:10:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295488+0000", 2021-01-15T19:10:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074145999999999999 2021-01-15T19:10:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296229+0000", 2021-01-15T19:10:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999 2021-01-15T19:10:07.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.414 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1117 3.4s1 3:21d37027:::smithi00140323-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e114)", 2021-01-15T19:10:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 25.002686518000001, 2021-01-15T19:10:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003, 2021-01-15T19:10:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1117 2021-01-15T19:10:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3238000000000002e-05 2021-01-15T19:10:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252749+0000", 2021-01-15T19:10:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8820000000000001e-06 2021-01-15T19:10:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252757+0000", 2021-01-15T19:10:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6419e-05 2021-01-15T19:10:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252774+0000", 2021-01-15T19:10:07.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9879999999999998e-06 2021-01-15T19:10:07.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252781+0000", 2021-01-15T19:10:07.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051378399999999999 2021-01-15T19:10:07.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253294+0000", 2021-01-15T19:10:07.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8572000000000006e-05 2021-01-15T19:10:07.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:07.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253373+0000", 2021-01-15T19:10:07.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040965935000000002 2021-01-15T19:10:07.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294339+0000", 2021-01-15T19:10:07.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8033000000000001e-05 2021-01-15T19:10:07.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:07.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294437+0000", 2021-01-15T19:10:07.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018291189999999999 2021-01-15T19:10:07.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296266+0000", 2021-01-15T19:10:07.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0880999999999999e-05 2021-01-15T19:10:07.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296357+0000", 2021-01-15T19:10:07.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016417 2021-01-15T19:10:07.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296521+0000", 2021-01-15T19:10:07.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003 2021-01-15T19:10:07.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.432 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.433 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s0 3.18aae252 (undecoded) ondisk+write+peerstat_old+known_if_redirected e119)", 2021-01-15T19:10:07.433 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:07.433 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.976050184999998, 2021-01-15T19:10:07.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263, 2021-01-15T19:10:07.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.434 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:10:07.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.435 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:07.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:07.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5269000000000003e-05 2021-01-15T19:10:07.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279388+0000", 2021-01-15T19:10:07.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4849999999999998e-06 2021-01-15T19:10:07.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279397+0000", 2021-01-15T19:10:07.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.63e-05 2021-01-15T19:10:07.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279414+0000", 2021-01-15T19:10:07.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:07.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279422+0000", 2021-01-15T19:10:07.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5129e-05 2021-01-15T19:10:07.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279477+0000", 2021-01-15T19:10:07.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1559999999999997e-06 2021-01-15T19:10:07.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:07.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279481+0000", 2021-01-15T19:10:07.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.92165986799999999 2021-01-15T19:10:07.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201141+0000", 2021-01-15T19:10:07.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9066999999999999e-05 2021-01-15T19:10:07.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201160+0000", 2021-01-15T19:10:07.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263 2021-01-15T19:10:07.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.446 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4166.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e99)", 2021-01-15T19:10:07.446 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:07.447 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.699680961999999, 2021-01-15T19:10:07.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116, 2021-01-15T19:10:07.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.447 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4166", 2021-01-15T19:10:07.448 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4191424063", 2021-01-15T19:10:07.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4 2021-01-15T19:10:07.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.448 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:07.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:07.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7705999999999999e-05 2021-01-15T19:10:07.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555739+0000", 2021-01-15T19:10:07.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8659999999999995e-06 2021-01-15T19:10:07.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555748+0000", 2021-01-15T19:10:07.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2021-01-15T19:10:07.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555761+0000", 2021-01-15T19:10:07.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9859999999999994e-06 2021-01-15T19:10:07.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555769+0000", 2021-01-15T19:10:07.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034585600000000002 2021-01-15T19:10:07.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556115+0000", 2021-01-15T19:10:07.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:07.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:07.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556129+0000", 2021-01-15T19:10:07.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.65062051300000001 2021-01-15T19:10:07.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206749+0000", 2021-01-15T19:10:07.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014163399999999999 2021-01-15T19:10:07.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206891+0000", 2021-01-15T19:10:07.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116 2021-01-15T19:10:07.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.459 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.460 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.4s1 124/122 ECSubWrite(tid=344, reqid=client.4418.0:1180, at_version=124'63, trim_to=0'0, roll_forward_to=123'61))", 2021-01-15T19:10:07.460 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:07.460 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.023180599, 2021-01-15T19:10:07.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011987622999999999, 2021-01-15T19:10:07.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:07.461 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:07.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:07.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6241e-05 2021-01-15T19:10:07.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232258+0000", 2021-01-15T19:10:07.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:07.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232273+0000", 2021-01-15T19:10:07.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2021-01-15T19:10:07.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232281+0000", 2021-01-15T19:10:07.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3479999999999997e-06 2021-01-15T19:10:07.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232285+0000", 2021-01-15T19:10:07.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2518999999999998e-05 2021-01-15T19:10:07.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232318+0000", 2021-01-15T19:10:07.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:07.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232325+0000", 2021-01-15T19:10:07.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011821569000000001 2021-01-15T19:10:07.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.244147+0000", 2021-01-15T19:10:07.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3057999999999999e-05 2021-01-15T19:10:07.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.244200+0000", 2021-01-15T19:10:07.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011987622999999999 2021-01-15T19:10:07.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.471 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.472 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.472 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1187 3.12s0 3:4bacf681:::smithi00140323-17:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:07.472 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:07.473 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.029007617, 2021-01-15T19:10:07.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999, 2021-01-15T19:10:07.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:07.473 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.474 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1187 2021-01-15T19:10:07.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.474 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:07.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:07.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8656999999999998e-05 2021-01-15T19:10:07.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226444+0000", 2021-01-15T19:10:07.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:07.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226452+0000", 2021-01-15T19:10:07.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3689000000000001e-05 2021-01-15T19:10:07.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226466+0000", 2021-01-15T19:10:07.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8540000000000002e-06 2021-01-15T19:10:07.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226472+0000", 2021-01-15T19:10:07.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2799999999999997e-05 2021-01-15T19:10:07.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226514+0000", 2021-01-15T19:10:07.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1725999999999999e-05 2021-01-15T19:10:07.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:07.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226526+0000", 2021-01-15T19:10:07.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066831169999999997 2021-01-15T19:10:07.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.233209+0000", 2021-01-15T19:10:07.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011647090000000001 2021-01-15T19:10:07.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234374+0000", 2021-01-15T19:10:07.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00061833799999999996 2021-01-15T19:10:07.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234992+0000", 2021-01-15T19:10:07.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001433157 2021-01-15T19:10:07.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.236425+0000", 2021-01-15T19:10:07.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044118559999999996 2021-01-15T19:10:07.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:07.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240837+0000", 2021-01-15T19:10:07.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152409 2021-01-15T19:10:07.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240990+0000", 2021-01-15T19:10:07.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999 2021-01-15T19:10:07.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.489 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.490 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1188 3.12s0 3:4bacf681:::smithi00140323-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:07.490 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:07.490 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.028872356999999, 2021-01-15T19:10:07.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001, 2021-01-15T19:10:07.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.491 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.491 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1188 2021-01-15T19:10:07.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.492 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:07.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:07.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0703e-05 2021-01-15T19:10:07.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226561+0000", 2021-01-15T19:10:07.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5660000000000001e-06 2021-01-15T19:10:07.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226569+0000", 2021-01-15T19:10:07.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3385e-05 2021-01-15T19:10:07.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226582+0000", 2021-01-15T19:10:07.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:07.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226587+0000", 2021-01-15T19:10:07.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.409e-05 2021-01-15T19:10:07.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226632+0000", 2021-01-15T19:10:07.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1242e-05 2021-01-15T19:10:07.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:07.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226643+0000", 2021-01-15T19:10:07.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086436099999999995 2021-01-15T19:10:07.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235286+0000", 2021-01-15T19:10:07.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016845599999999999 2021-01-15T19:10:07.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:07.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235455+0000", 2021-01-15T19:10:07.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060319140000000002 2021-01-15T19:10:07.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241487+0000", 2021-01-15T19:10:07.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014580699999999999 2021-01-15T19:10:07.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241633+0000", 2021-01-15T19:10:07.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001527306 2021-01-15T19:10:07.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.243160+0000", 2021-01-15T19:10:07.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001 2021-01-15T19:10:07.505 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.506 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.506 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1192 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~557056] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9997682720000007, 2021-01-15T19:10:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012911791000000001, 2021-01-15T19:10:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:07.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.508 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1192 2021-01-15T19:10:07.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.509 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:07.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:07.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042538899999999998 2021-01-15T19:10:07.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256050+0000", 2021-01-15T19:10:07.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1664999999999999e-05 2021-01-15T19:10:07.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256061+0000", 2021-01-15T19:10:07.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0514e-05 2021-01-15T19:10:07.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256082+0000", 2021-01-15T19:10:07.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0865e-05 2021-01-15T19:10:07.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256093+0000", 2021-01-15T19:10:07.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2021-01-15T19:10:07.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256160+0000", 2021-01-15T19:10:07.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013985700000000001 2021-01-15T19:10:07.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256300+0000", 2021-01-15T19:10:07.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001360053 2021-01-15T19:10:07.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.257660+0000", 2021-01-15T19:10:07.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010289375999999999 2021-01-15T19:10:07.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267950+0000", 2021-01-15T19:10:07.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051317300000000001 2021-01-15T19:10:07.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:07.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268463+0000", 2021-01-15T19:10:07.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3504999999999995e-05 2021-01-15T19:10:07.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268536+0000", 2021-01-15T19:10:07.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012911791000000001 2021-01-15T19:10:07.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.521 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.522 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1193 3.12s0 3:4e4266df:::smithi00140323-37:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:07.522 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:07.522 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9992510049999996, 2021-01-15T19:10:07.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013858986, 2021-01-15T19:10:07.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:07.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1193 2021-01-15T19:10:07.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.524 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:07.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:07.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7895999999999997e-05 2021-01-15T19:10:07.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256190+0000", 2021-01-15T19:10:07.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5399999999999998e-06 2021-01-15T19:10:07.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256197+0000", 2021-01-15T19:10:07.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.252e-05 2021-01-15T19:10:07.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256210+0000", 2021-01-15T19:10:07.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1340000000000001e-06 2021-01-15T19:10:07.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256216+0000", 2021-01-15T19:10:07.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010534175 2021-01-15T19:10:07.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.266750+0000", 2021-01-15T19:10:07.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109787 2021-01-15T19:10:07.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.266860+0000", 2021-01-15T19:10:07.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00057329899999999999 2021-01-15T19:10:07.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:07.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267433+0000", 2021-01-15T19:10:07.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076203700000000002 2021-01-15T19:10:07.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:07.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268195+0000", 2021-01-15T19:10:07.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001719424 2021-01-15T19:10:07.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:07.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.269914+0000", 2021-01-15T19:10:07.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6174000000000005e-05 2021-01-15T19:10:07.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270001+0000", 2021-01-15T19:10:07.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013858986 2021-01-15T19:10:07.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1194 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9991293979999991, 2021-01-15T19:10:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019, 2021-01-15T19:10:07.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:07.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1194 2021-01-15T19:10:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8844999999999998e-05 2021-01-15T19:10:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256302+0000", 2021-01-15T19:10:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9030000000000004e-06 2021-01-15T19:10:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256310+0000", 2021-01-15T19:10:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4749999999999999e-05 2021-01-15T19:10:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256325+0000", 2021-01-15T19:10:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2370000000000003e-06 2021-01-15T19:10:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256332+0000", 2021-01-15T19:10:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011409671999999999 2021-01-15T19:10:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267742+0000", 2021-01-15T19:10:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017902200000000001 2021-01-15T19:10:07.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267921+0000", 2021-01-15T19:10:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027501040000000002 2021-01-15T19:10:07.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270671+0000", 2021-01-15T19:10:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020387700000000001 2021-01-15T19:10:07.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270875+0000", 2021-01-15T19:10:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012356089999999999 2021-01-15T19:10:07.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.272110+0000", 2021-01-15T19:10:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019 2021-01-15T19:10:07.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:07.552 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:07.552 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:07.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:07.557 INFO:tasks.rados.rados.0.smithi001.stdout:547: done (2 left) 2021-01-15T19:10:07.557 INFO:tasks.rados.rados.0.smithi001.stdout:551: done (1 left) 2021-01-15T19:10:07.557 INFO:tasks.rados.rados.0.smithi001.stdout:552: done (0 left) 2021-01-15T19:10:07.557 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 16 current snap is 61 2021-01-15T19:10:07.558 INFO:tasks.rados.rados.0.smithi001.stdout:553: seq_num 197 ranges {0=507904,507904=401408} 2021-01-15T19:10:07.558 INFO:tasks.rados.rados.0.smithi001.stdout:553: writing smithi00140323-16 from 0 to 507904 tid 1 2021-01-15T19:10:07.558 INFO:tasks.rados.rados.0.smithi001.stdout:553: writing smithi00140323-16 from 507904 to 909312 tid 2 2021-01-15T19:10:07.558 INFO:tasks.rados.rados.0.smithi001.stdout:554: delete oid 24 current snap is 61 2021-01-15T19:10:07.558 INFO:tasks.rados.rados.0.smithi001.stdout:553: finishing write tid 1 to smithi00140323-16 2021-01-15T19:10:07.559 INFO:tasks.rados.rados.0.smithi001.stdout:553: finishing write tid 2 to smithi00140323-16 2021-01-15T19:10:07.559 INFO:tasks.rados.rados.0.smithi001.stdout:553: finishing write tid 3 to smithi00140323-16 2021-01-15T19:10:07.559 INFO:tasks.rados.rados.0.smithi001.stdout:554: done (1 left) 2021-01-15T19:10:07.559 INFO:tasks.rados.rados.0.smithi001.stdout:555: delete oid 45 current snap is 61 2021-01-15T19:10:07.559 INFO:tasks.rados.rados.0.smithi001.stdout:553: finishing write tid 4 to smithi00140323-16 2021-01-15T19:10:07.560 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 145 (ObjNum 197 snap 61 seq_num 197) dirty exists 2021-01-15T19:10:07.560 INFO:tasks.rados.rados.0.smithi001.stdout:553: left oid 16 (ObjNum 197 snap 61 seq_num 197) 2021-01-15T19:10:07.560 INFO:tasks.rados.rados.0.smithi001.stdout:553: done (1 left) 2021-01-15T19:10:07.560 INFO:tasks.rados.rados.0.smithi001.stdout:555: done (0 left) 2021-01-15T19:10:07.560 INFO:tasks.rados.rados.0.smithi001.stdout:556: rollback oid 46 current snap is 61 2021-01-15T19:10:07.561 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 46 to 57 2021-01-15T19:10:07.561 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 2 current snap is 61 2021-01-15T19:10:07.561 INFO:tasks.rados.rados.0.smithi001.stdout:557: seq_num 200 ranges {2613248=557056,3170304=401408} 2021-01-15T19:10:07.561 INFO:tasks.rados.rados.0.smithi001.stdout:557: writing smithi00140323-2 from 2613248 to 3170304 tid 1 2021-01-15T19:10:07.561 INFO:tasks.rados.rados.0.smithi001.stdout:557: writing smithi00140323-2 from 3170304 to 3571712 tid 2 2021-01-15T19:10:07.562 INFO:tasks.rados.rados.0.smithi001.stdout:558: read oid 3 snap -1 2021-01-15T19:10:07.562 INFO:tasks.rados.rados.0.smithi001.stdout:558: expect (ObjNum 170 snap 49 seq_num 170) 2021-01-15T19:10:07.562 INFO:tasks.rados.rados.0.smithi001.stdout:559: snap_create 2021-01-15T19:10:07.562 INFO:tasks.rados.rados.0.smithi001.stdout:556: finishing rollback tid 0 to smithi00140323-46 2021-01-15T19:10:07.562 INFO:tasks.rados.rados.0.smithi001.stdout:556: finishing rollback tid 1 to smithi00140323-46 2021-01-15T19:10:07.563 INFO:tasks.rados.rados.0.smithi001.stdout:556: finishing rollback tid 2 to smithi00140323-46 2021-01-15T19:10:07.563 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 46 v 64 (ObjNum 45 snap 0 seq_num 45) dirty exists 2021-01-15T19:10:07.563 INFO:tasks.rados.rados.0.smithi001.stdout:558: expect (ObjNum 170 snap 49 seq_num 170) 2021-01-15T19:10:07.563 INFO:tasks.rados.rados.0.smithi001.stdout:557: finishing write tid 1 to smithi00140323-2 2021-01-15T19:10:07.563 INFO:tasks.rados.rados.0.smithi001.stdout:557: finishing write tid 2 to smithi00140323-2 2021-01-15T19:10:07.564 INFO:tasks.rados.rados.0.smithi001.stdout:557: finishing write tid 3 to smithi00140323-2 2021-01-15T19:10:07.564 INFO:tasks.rados.rados.0.smithi001.stdout:557: finishing write tid 4 to smithi00140323-2 2021-01-15T19:10:07.564 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 155 (ObjNum 200 snap 61 seq_num 200) dirty exists 2021-01-15T19:10:07.564 INFO:tasks.rados.rados.0.smithi001.stdout:557: left oid 2 (ObjNum 200 snap 61 seq_num 200) 2021-01-15T19:10:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:10:07.565 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:07.553+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:07.655 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:07.654+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:07.692 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:07.728 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:07.728 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:07.728 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:07.729 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:07.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:07.779 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:07.778+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:07.841 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:07.857 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:07.856+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:07.862 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:07.882 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:07.903 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:07.914 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:07.914 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:07.924 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:07.925 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:07.945 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:07.958 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:07.956+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:07.966 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:07.987 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:07.989 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.989 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.989 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.989 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.990 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.990 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.990 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:07.990 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:08.057 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:08.057 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:08.060 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:08.059+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:08.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:08.161 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:08.160+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:08.198 INFO:tasks.rados.rados.0.smithi001.stdout:556: done (3 left) 2021-01-15T19:10:08.198 INFO:tasks.rados.rados.0.smithi001.stdout:557: done (2 left) 2021-01-15T19:10:08.198 INFO:tasks.rados.rados.0.smithi001.stdout:558: done (1 left) 2021-01-15T19:10:08.199 INFO:tasks.rados.rados.0.smithi001.stdout:559: done (0 left) 2021-01-15T19:10:08.199 INFO:tasks.rados.rados.0.smithi001.stdout:560: rmattr oid 4 current snap is 62 2021-01-15T19:10:08.199 INFO:tasks.rados.rados.0.smithi001.stdout:560: done (0 left) 2021-01-15T19:10:08.199 INFO:tasks.rados.rados.0.smithi001.stdout:561: rollback oid 15 current snap is 62 2021-01-15T19:10:08.199 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 15 to 59 2021-01-15T19:10:08.200 INFO:tasks.rados.rados.0.smithi001.stdout:562: copy_from oid 42 from oid 48 current snap is 62 2021-01-15T19:10:08.200 INFO:tasks.rados.rados.0.smithi001.stdout:563: rmattr oid 18 current snap is 62 2021-01-15T19:10:08.200 INFO:tasks.rados.rados.0.smithi001.stdout:563: done (2 left) 2021-01-15T19:10:08.200 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 62 2021-01-15T19:10:08.200 INFO:tasks.rados.rados.0.smithi001.stdout:564: seq_num 201 ranges {385024=573440,958464=401408} 2021-01-15T19:10:08.203 INFO:tasks.rados.rados.0.smithi001.stdout:564: writing smithi00140323-14 from 385024 to 958464 tid 1 2021-01-15T19:10:08.206 INFO:tasks.rados.rados.0.smithi001.stdout:564: writing smithi00140323-14 from 958464 to 1359872 tid 2 2021-01-15T19:10:08.207 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 62 2021-01-15T19:10:08.207 INFO:tasks.rados.rados.0.smithi001.stdout:561: finishing rollback tid 1 to smithi00140323-15 2021-01-15T19:10:08.207 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 15 v 76 (ObjNum 179087888 snap 22026 seq_num 179025024) dirty dne 2021-01-15T19:10:08.207 INFO:tasks.rados.rados.0.smithi001.stdout:565: seq_num 202 ranges {7512064=589824,8101888=401408} 2021-01-15T19:10:08.208 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:08.208 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:08.231 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:08.254 INFO:tasks.rados.rados.0.smithi001.stdout:565: writing smithi00140323-13 from 7512064 to 8101888 tid 1 2021-01-15T19:10:08.257 INFO:tasks.rados.rados.0.smithi001.stdout:565: writing smithi00140323-13 from 8101888 to 8503296 tid 2 2021-01-15T19:10:08.257 INFO:tasks.rados.rados.0.smithi001.stdout:561: done (3 left) 2021-01-15T19:10:08.258 INFO:tasks.rados.rados.0.smithi001.stdout:566: copy_from oid 37 from oid 33 current snap is 62 2021-01-15T19:10:08.258 INFO:tasks.rados.rados.0.smithi001.stdout:564: finishing write tid 1 to smithi00140323-14 2021-01-15T19:10:08.258 INFO:tasks.rados.rados.0.smithi001.stdout:564: finishing write tid 2 to smithi00140323-14 2021-01-15T19:10:08.258 INFO:tasks.rados.rados.0.smithi001.stdout:564: finishing write tid 3 to smithi00140323-14 2021-01-15T19:10:08.259 INFO:tasks.rados.rados.0.smithi001.stdout:567: read oid 50 snap -1 2021-01-15T19:10:08.259 INFO:tasks.rados.rados.0.smithi001.stdout:567: expect (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:10:08.259 INFO:tasks.rados.rados.0.smithi001.stdout:568: read oid 8 snap -1 2021-01-15T19:10:08.259 INFO:tasks.rados.rados.0.smithi001.stdout:568: expect (ObjNum 188 snap 58 seq_num 188) 2021-01-15T19:10:08.259 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 17 current snap is 62 2021-01-15T19:10:08.260 INFO:tasks.rados.rados.0.smithi001.stdout:564: finishing write tid 4 to smithi00140323-14 2021-01-15T19:10:08.260 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 91 (ObjNum 201 snap 62 seq_num 201) dirty exists 2021-01-15T19:10:08.260 INFO:tasks.rados.rados.0.smithi001.stdout:564: left oid 14 (ObjNum 201 snap 62 seq_num 201) 2021-01-15T19:10:08.260 INFO:tasks.rados.rados.0.smithi001.stdout:562: finishing copy_from to smithi00140323-42 2021-01-15T19:10:08.260 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 148 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:10:08.261 INFO:tasks.rados.rados.0.smithi001.stdout:562: finishing copy_from racing read to smithi00140323-42 2021-01-15T19:10:08.261 INFO:tasks.rados.rados.0.smithi001.stdout:569: seq_num 203 ranges {540672=606208,1146880=401408} 2021-01-15T19:10:08.263 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:08.262+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:08.263 INFO:tasks.rados.rados.0.smithi001.stdout:569: writing smithi00140323-17 from 540672 to 1146880 tid 1 2021-01-15T19:10:08.266 INFO:tasks.rados.rados.0.smithi001.stdout:569: writing smithi00140323-17 from 1146880 to 1548288 tid 2 2021-01-15T19:10:08.266 INFO:tasks.rados.rados.0.smithi001.stdout:562: done (6 left) 2021-01-15T19:10:08.266 INFO:tasks.rados.rados.0.smithi001.stdout:564: done (5 left) 2021-01-15T19:10:08.267 INFO:tasks.rados.rados.0.smithi001.stdout:570: snap_create 2021-01-15T19:10:08.267 INFO:tasks.rados.rados.0.smithi001.stdout:566: finishing copy_from to smithi00140323-37 2021-01-15T19:10:08.267 INFO:tasks.rados.rados.0.smithi001.stdout:566: got expected ENOENT (src dne) 2021-01-15T19:10:08.268 INFO:tasks.rados.rados.0.smithi001.stdout:566: finishing copy_from racing read to smithi00140323-37 2021-01-15T19:10:08.269 INFO:tasks.rados.rados.0.smithi001.stdout:567: expect (ObjNum 53 snap 1 seq_num 53) 2021-01-15T19:10:08.340 INFO:tasks.rados.rados.0.smithi001.stdout:565: finishing write tid 1 to smithi00140323-13 2021-01-15T19:10:08.340 INFO:tasks.rados.rados.0.smithi001.stdout:569: finishing write tid 1 to smithi00140323-17 2021-01-15T19:10:08.340 INFO:tasks.rados.rados.0.smithi001.stdout:569: finishing write tid 2 to smithi00140323-17 2021-01-15T19:10:08.341 INFO:tasks.rados.rados.0.smithi001.stdout:565: finishing write tid 2 to smithi00140323-13 2021-01-15T19:10:08.341 INFO:tasks.rados.rados.0.smithi001.stdout:569: finishing write tid 3 to smithi00140323-17 2021-01-15T19:10:08.341 INFO:tasks.rados.rados.0.smithi001.stdout:565: finishing write tid 3 to smithi00140323-13 2021-01-15T19:10:08.341 INFO:tasks.rados.rados.0.smithi001.stdout:569: finishing write tid 4 to smithi00140323-17 2021-01-15T19:10:08.341 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 17 v 118 (ObjNum 203 snap 62 seq_num 203) dirty exists 2021-01-15T19:10:08.342 INFO:tasks.rados.rados.0.smithi001.stdout:569: left oid 17 (ObjNum 203 snap 62 seq_num 203) 2021-01-15T19:10:08.342 INFO:tasks.rados.rados.0.smithi001.stdout:565: finishing write tid 4 to smithi00140323-13 2021-01-15T19:10:08.342 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 248 (ObjNum 202 snap 62 seq_num 202) dirty exists 2021-01-15T19:10:08.342 INFO:tasks.rados.rados.0.smithi001.stdout:565: left oid 13 (ObjNum 202 snap 62 seq_num 202) 2021-01-15T19:10:08.343 INFO:tasks.rados.rados.0.smithi001.stdout:568: expect (ObjNum 188 snap 58 seq_num 188) 2021-01-15T19:10:08.346 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:08.346 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:10:08.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2021-01-15T19:10:08.382 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:08.382 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:08.383 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:08.381+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:08.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:08.466 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:08.465+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:08.571 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:08.570+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:08.582 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:08.582 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:08.592 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:08.672 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:08.671+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:08.686 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:10:08.687 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:08.687 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 140, 2021-01-15T19:10:08.687 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "fe0e1890-043f-40bf-ad11-c5f08349f418", 2021-01-15T19:10:08.687 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2021-01-15T19:07:43.378003+0000", 2021-01-15T19:10:08.688 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2021-01-15T19:10:08.192363+0000", 2021-01-15T19:10:08.688 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2021-01-15T19:10:00.310252+0000", 2021-01-15T19:10:08.688 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2021-01-15T19:09:47.269026+0000", 2021-01-15T19:10:08.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-01-15T19:10:08.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5801984, 2021-01-15T19:10:08.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2021-01-15T19:10:08.689 INFO:teuthology.orchestra.run.smithi001.stdout: "noscrub", 2021-01-15T19:10:08.689 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2021-01-15T19:10:08.689 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2021-01-15T19:10:08.690 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2021-01-15T19:10:08.690 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2021-01-15T19:10:08.690 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:10:08.690 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2021-01-15T19:10:08.690 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2021-01-15T19:10:08.691 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-01-15T19:10:08.691 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-01-15T19:10:08.691 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2021-01-15T19:10:08.691 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2021-01-15T19:10:08.691 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2021-01-15T19:10:08.692 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2021-01-15T19:10:08.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "jewel", 2021-01-15T19:10:08.692 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "pacific", 2021-01-15T19:10:08.692 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2021-01-15T19:10:08.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.693 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2021-01-15T19:10:08.693 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "device_health_metrics", 2021-01-15T19:10:08.693 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:47.545867+0000", 2021-01-15T19:10:08.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:10:08.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:10:08.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:10:08.694 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:10:08.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:10:08.695 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:10:08.695 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:10:08.695 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:10:08.695 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:10:08.695 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:10:08.696 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:10:08.696 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:10:08.696 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 11, 2021-01-15T19:10:08.696 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 11, 2021-01-15T19:10:08.696 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 11, 2021-01-15T19:10:08.697 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 11, 2021-01-15T19:10:08.697 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 11, 2021-01-15T19:10:08.697 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:10:08.697 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:10:08.697 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:10:08.698 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:10:08.698 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:10:08.698 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:10:08.698 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:10:08.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "113", 2021-01-15T19:10:08.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:10:08.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:10:08.699 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "58", 2021-01-15T19:10:08.700 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:10:08.700 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:10:08.700 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:10:08.700 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:10:08.700 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:10:08.701 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:10:08.701 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:10:08.701 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:10:08.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:10:08.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:10:08.702 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:10:08.702 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:10:08.702 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:10:08.702 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:10:08.702 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:10:08.703 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:10:08.703 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:10:08.703 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:10:08.703 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:10:08.703 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:10:08.704 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:10:08.704 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:10:08.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:10:08.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.705 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:10:08.705 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:10:08.705 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:10:08.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:10:08.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:10:08.706 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:10:08.706 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:10:08.706 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:10:08.706 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:10:08.707 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:10:08.707 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:10:08.707 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2021-01-15T19:10:08.707 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2021-01-15T19:10:08.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.708 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:10:08.708 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_devicehealth": {} 2021-01-15T19:10:08.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.709 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2021-01-15T19:10:08.709 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2021-01-15T19:10:08.709 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:54.484638+0000", 2021-01-15T19:10:08.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:10:08.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:10:08.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:10:08.710 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:10:08.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:10:08.710 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:10:08.710 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:10:08.711 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:10:08.711 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:10:08.711 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:10:08.711 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:10:08.712 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:10:08.712 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 18, 2021-01-15T19:10:08.712 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 18, 2021-01-15T19:10:08.712 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 18, 2021-01-15T19:10:08.712 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 18, 2021-01-15T19:10:08.713 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 18, 2021-01-15T19:10:08.713 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:10:08.713 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:10:08.713 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:10:08.713 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:10:08.714 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:10:08.714 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:10:08.714 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:10:08.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.715 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "48", 2021-01-15T19:10:08.715 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:10:08.715 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:10:08.715 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "46", 2021-01-15T19:10:08.715 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:10:08.716 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:10:08.716 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:10:08.716 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:10:08.716 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:10:08.717 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:10:08.717 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:10:08.717 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:10:08.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:10:08.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:10:08.718 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:10:08.718 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:10:08.718 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:10:08.718 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:10:08.718 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:10:08.719 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:10:08.719 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:10:08.719 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:10:08.719 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:10:08.719 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:10:08.720 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:10:08.720 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:10:08.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:10:08.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.720 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:10:08.721 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:10:08.721 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:10:08.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:10:08.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:10:08.721 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:10:08.722 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:10:08.722 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:10:08.722 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:10:08.722 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:10:08.722 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:10:08.723 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:10:08.723 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:10:08.723 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2021-01-15T19:10:08.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.724 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:10:08.724 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "unique_pool_0", 2021-01-15T19:10:08.724 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:08:08.242953+0000", 2021-01-15T19:10:08.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2021-01-15T19:10:08.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2021-01-15T19:10:08.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 3, 2021-01-15T19:10:08.725 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 3, 2021-01-15T19:10:08.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 2, 2021-01-15T19:10:08.726 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 1, 2021-01-15T19:10:08.726 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:10:08.726 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:10:08.726 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:10:08.727 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:10:08.727 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:10:08.727 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:10:08.727 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 26, 2021-01-15T19:10:08.727 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 26, 2021-01-15T19:10:08.728 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 26, 2021-01-15T19:10:08.728 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 26, 2021-01-15T19:10:08.728 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 26, 2021-01-15T19:10:08.728 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:10:08.728 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:10:08.729 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:10:08.729 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:10:08.729 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:10:08.729 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:10:08.729 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:10:08.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.730 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "140", 2021-01-15T19:10:08.730 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:10:08.730 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:10:08.730 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "52", 2021-01-15T19:10:08.731 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:10:08.731 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:10:08.731 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 111, 2021-01-15T19:10:08.731 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 140, 2021-01-15T19:10:08.731 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:10:08.732 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:10:08.732 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:10:08.732 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:10:08.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:10:08.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:10:08.733 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:10:08.733 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:10:08.733 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:10:08.733 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:10:08.734 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:10:08.734 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:10:08.734 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:10:08.734 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:10:08.734 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:10:08.735 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:10:08.735 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "isaprofile", 2021-01-15T19:10:08.735 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:10:08.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:10:08.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:10:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:10:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:10:08.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:10:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:10:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:10:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:10:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:10:08.737 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 8192, 2021-01-15T19:10:08.738 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:10:08.738 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:10:08.738 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:10:08.738 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:10:08.738 INFO:teuthology.orchestra.run.smithi001.stdout: "rados": {} 2021-01-15T19:10:08.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.739 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.739 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:10:08.739 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:10:08.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.740 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:10:08.740 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e5b26f89-5c99-4764-9183-f3bb27c118f5", 2021-01-15T19:10:08.740 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.740 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:10:08.740 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:10:08.741 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.741 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.741 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.741 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.741 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 122, 2021-01-15T19:10:08.742 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.742 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.742 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.742 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2021-01-15T19:10:08.743 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.744 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2021-01-15T19:10:08.744 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.745 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.745 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2021-01-15T19:10:08.746 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2021-01-15T19:10:08.747 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.748 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2021-01-15T19:10:08.749 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.749 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2021-01-15T19:10:08.750 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.750 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.751 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.751 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2021-01-15T19:10:08.751 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.752 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2021-01-15T19:10:08.752 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:10:08.753 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.753 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.753 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/34833", 2021-01-15T19:10:08.753 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/34833", 2021-01-15T19:10:08.754 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/34833", 2021-01-15T19:10:08.754 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/34833", 2021-01-15T19:10:08.754 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.754 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.754 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.755 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:10:08.755 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "99c0d455-1bd6-43aa-bfec-b6bddf87a35e", 2021-01-15T19:10:08.756 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.756 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:10:08.756 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:10:08.756 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.756 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 122, 2021-01-15T19:10:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.758 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.758 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.758 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2021-01-15T19:10:08.759 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.759 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2021-01-15T19:10:08.760 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.760 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.760 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.760 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.761 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.761 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2021-01-15T19:10:08.762 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.762 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2021-01-15T19:10:08.763 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.763 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.764 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.764 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2021-01-15T19:10:08.764 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.765 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2021-01-15T19:10:08.765 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.766 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.766 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.766 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.767 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2021-01-15T19:10:08.767 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.768 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2021-01-15T19:10:08.768 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:10:08.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.769 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.769 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/34834", 2021-01-15T19:10:08.769 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/34834", 2021-01-15T19:10:08.769 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/34834", 2021-01-15T19:10:08.770 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/34834", 2021-01-15T19:10:08.770 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.770 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.770 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.771 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.771 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:10:08.771 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "71fe6c4b-3d73-4e07-b49d-98596ca7d9da", 2021-01-15T19:10:08.772 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.772 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:10:08.772 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:10:08.772 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.772 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.773 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.773 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.773 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 122, 2021-01-15T19:10:08.773 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.773 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.774 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.774 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.774 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2021-01-15T19:10:08.775 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.775 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2021-01-15T19:10:08.776 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.776 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.776 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.777 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.777 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2021-01-15T19:10:08.777 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.778 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2021-01-15T19:10:08.779 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.779 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.779 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.780 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.780 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2021-01-15T19:10:08.780 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.781 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2021-01-15T19:10:08.781 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.782 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.782 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.782 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.782 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.783 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2021-01-15T19:10:08.783 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.784 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2021-01-15T19:10:08.784 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:10:08.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.785 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6819/34863", 2021-01-15T19:10:08.785 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6823/34863", 2021-01-15T19:10:08.786 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/34863", 2021-01-15T19:10:08.786 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/34863", 2021-01-15T19:10:08.786 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.786 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.786 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.787 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.787 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:10:08.787 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8a6b018a-3a48-4a01-8526-a386f62ecb36", 2021-01-15T19:10:08.788 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.788 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:10:08.788 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0.198638916015625, 2021-01-15T19:10:08.788 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.788 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.789 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.789 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.789 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 119, 2021-01-15T19:10:08.789 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.789 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.790 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.790 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2021-01-15T19:10:08.791 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.792 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2021-01-15T19:10:08.792 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.793 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.793 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.793 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2021-01-15T19:10:08.794 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.794 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2021-01-15T19:10:08.795 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.795 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.795 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.796 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2021-01-15T19:10:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.797 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2021-01-15T19:10:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.798 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2021-01-15T19:10:08.799 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2021-01-15T19:10:08.800 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:10:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6818/34864", 2021-01-15T19:10:08.801 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6822/34864", 2021-01-15T19:10:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/34864", 2021-01-15T19:10:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/34864", 2021-01-15T19:10:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.802 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:10:08.803 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "ff089622-e642-4bd0-901c-60efdb87a549", 2021-01-15T19:10:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:10:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:10:08.804 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 122, 2021-01-15T19:10:08.805 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6800", 2021-01-15T19:10:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6801", 2021-01-15T19:10:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.809 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6802", 2021-01-15T19:10:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.810 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6803", 2021-01-15T19:10:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6806", 2021-01-15T19:10:08.812 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6807", 2021-01-15T19:10:08.813 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.814 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6804", 2021-01-15T19:10:08.815 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6805", 2021-01-15T19:10:08.816 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:10:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6801/34829", 2021-01-15T19:10:08.817 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6803/34829", 2021-01-15T19:10:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6807/34829", 2021-01-15T19:10:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6805/34829", 2021-01-15T19:10:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.818 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:10:08.819 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "3c57c228-dfaa-4e33-bded-27da0048d141", 2021-01-15T19:10:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 0, 2021-01-15T19:10:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2021-01-15T19:10:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.820 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 86, 2021-01-15T19:10:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.821 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.822 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6809", 2021-01-15T19:10:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6811", 2021-01-15T19:10:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.824 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6813", 2021-01-15T19:10:08.825 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6815", 2021-01-15T19:10:08.826 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.827 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6821", 2021-01-15T19:10:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6823", 2021-01-15T19:10:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6817", 2021-01-15T19:10:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6819", 2021-01-15T19:10:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:10:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.832 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6811/34830", 2021-01-15T19:10:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6815/34830", 2021-01-15T19:10:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6823/34830", 2021-01-15T19:10:08.833 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6819/34830", 2021-01-15T19:10:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:10:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "201f889f-dac1-4a26-a285-7a0ec21c3646", 2021-01-15T19:10:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.835 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 0, 2021-01-15T19:10:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2021-01-15T19:10:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 13, 2021-01-15T19:10:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 94, 2021-01-15T19:10:08.836 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 132, 2021-01-15T19:10:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 92, 2021-01-15T19:10:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 95, 2021-01-15T19:10:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.837 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6808", 2021-01-15T19:10:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6810", 2021-01-15T19:10:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6812", 2021-01-15T19:10:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6814", 2021-01-15T19:10:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.842 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6820", 2021-01-15T19:10:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6822", 2021-01-15T19:10:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.846 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6816", 2021-01-15T19:10:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.847 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6818", 2021-01-15T19:10:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:10:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.848 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6810/48096", 2021-01-15T19:10:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6814/48096", 2021-01-15T19:10:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6822/48096", 2021-01-15T19:10:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6818/48096", 2021-01-15T19:10:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.849 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.850 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:10:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "79ffd413-a9d3-4180-9691-d834d89c3f2e", 2021-01-15T19:10:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:10:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:10:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:10:08.851 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:10:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:10:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:10:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:10:08.852 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 122, 2021-01-15T19:10:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:10:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:10:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:10:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6824", 2021-01-15T19:10:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6825", 2021-01-15T19:10:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:10:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6826", 2021-01-15T19:10:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6827", 2021-01-15T19:10:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:10:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.859 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6830", 2021-01-15T19:10:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.860 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6831", 2021-01-15T19:10:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.861 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:10:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:10:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:10:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6828", 2021-01-15T19:10:08.862 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:10:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6829", 2021-01-15T19:10:08.863 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:10:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.864 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6825/34874", 2021-01-15T19:10:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6827/34874", 2021-01-15T19:10:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6831/34874", 2021-01-15T19:10:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6829/34874", 2021-01-15T19:10:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:10:08.865 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:10:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:10:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:10:08.866 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2021-01-15T19:10:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:10:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.867 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:10:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:49.068217+0000", 2021-01-15T19:10:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:10:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.869 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:10:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:49.114373+0000", 2021-01-15T19:10:08.870 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:10:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.871 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:10:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:50.647382+0000", 2021-01-15T19:10:08.872 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:10:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.873 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:10:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:49.296499+0000", 2021-01-15T19:10:08.874 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:10:08.875 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:10:08.876 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:50.239164+0000", 2021-01-15T19:10:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:10:08.877 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 65536, 2021-01-15T19:10:08.878 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:50.384906+0000", 2021-01-15T19:10:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:10:08.879 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 65536, 2021-01-15T19:10:08.880 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:10:00.995180+0000", 2021-01-15T19:10:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 95 2021-01-15T19:10:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:10:08.881 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:10:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:10:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:10:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:10:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:10:08.882 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:50.678303+0000", 2021-01-15T19:10:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:10:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:10:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2021-01-15T19:10:08.883 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2021-01-15T19:10:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [], 2021-01-15T19:10:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2021-01-15T19:10:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2021-01-15T19:10:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2021-01-15T19:10:08.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2021-01-15T19:10:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:10:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:10:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:10:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2021-01-15T19:10:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "ruleset-failure-domain": "osd", 2021-01-15T19:10:08.885 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:10:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "isaprofile": { 2021-01-15T19:10:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-device-class": "", 2021-01-15T19:10:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:10:08.886 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-root": "default", 2021-01-15T19:10:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:10:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:10:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "isaprofile", 2021-01-15T19:10:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "isa", 2021-01-15T19:10:08.887 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:10:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [ 2021-01-15T19:10:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.888 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:10:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2021-01-15T19:10:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 25, 2021-01-15T19:10:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 42, 2021-01-15T19:10:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 47, 2021-01-15T19:10:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.891 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 56, 2021-01-15T19:10:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 58, 2021-01-15T19:10:08.892 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 62, 2021-01-15T19:10:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 65, 2021-01-15T19:10:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.895 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 70, 2021-01-15T19:10:08.895 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.895 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 76, 2021-01-15T19:10:08.896 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.896 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 80, 2021-01-15T19:10:08.896 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.897 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 83, 2021-01-15T19:10:08.897 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:10:08.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.898 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 87, 2021-01-15T19:10:08.898 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 3 2021-01-15T19:10:08.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.899 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 92, 2021-01-15T19:10:08.899 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:10:08.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:08.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:08.899 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 98, 2021-01-15T19:10:08.900 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:10:08.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.900 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:08.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.900 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:10:08.901 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2021-01-15T19:10:08.901 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2021-01-15T19:10:08.901 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2021-01-15T19:10:08.901 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2021-01-15T19:10:08.901 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2021-01-15T19:10:08.902 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2021-01-15T19:10:08.902 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2021-01-15T19:10:08.902 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2021-01-15T19:10:08.902 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2021-01-15T19:10:08.902 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2021-01-15T19:10:08.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:08.903 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:08.905 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:08.905 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:08.905 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:08.903+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:08.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:08.907 INFO:tasks.thrashosds.thrasher:j is {'epoch': 140, 'fsid': 'fe0e1890-043f-40bf-ad11-c5f08349f418', 'created': '2021-01-15T19:07:43.378003+0000', 'modified': '2021-01-15T19:10:08.192363+0000', 'last_up_change': '2021-01-15T19:10:00.310252+0000', 'last_in_change': '2021-01-15T19:09:47.269026+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '58', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 18, 'pg_placement_num': 18, 'pg_placement_num_target': 18, 'pg_num_target': 18, 'pg_num_pending': 18, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '46', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '52', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 111, 'snap_epoch': 140, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e5b26f89-5c99-4764-9183-f3bb27c118f5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 34833}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 34833}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 34833}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 34833}]}, 'public_addr': '172.21.15.1:6801/34833', 'cluster_addr': '172.21.15.1:6803/34833', 'heartbeat_back_addr': '172.21.15.1:6807/34833', 'heartbeat_front_addr': '172.21.15.1:6805/34833', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '99c0d455-1bd6-43aa-bfec-b6bddf87a35e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 34834}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 34834}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 34834}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 34834}]}, 'public_addr': '172.21.15.1:6809/34834', 'cluster_addr': '172.21.15.1:6811/34834', 'heartbeat_back_addr': '172.21.15.1:6815/34834', 'heartbeat_front_addr': '172.21.15.1:6813/34834', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '71fe6c4b-3d73-4e07-b49d-98596ca7d9da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6817', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 34863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6821', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 34863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 34863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 34863}]}, 'public_addr': '172.21.15.1:6819/34863', 'cluster_addr': '172.21.15.1:6823/34863', 'heartbeat_back_addr': '172.21.15.1:6831/34863', 'heartbeat_front_addr': '172.21.15.1:6828/34863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8a6b018a-3a48-4a01-8526-a386f62ecb36', 'up': 1, 'in': 1, 'weight': 0.198638916015625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 119, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6818', 'nonce': 34864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6822', 'nonce': 34864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 34864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 34864}]}, 'public_addr': '172.21.15.1:6818/34864', 'cluster_addr': '172.21.15.1:6822/34864', 'heartbeat_back_addr': '172.21.15.1:6829/34864', 'heartbeat_front_addr': '172.21.15.1:6825/34864', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ff089622-e642-4bd0-901c-60efdb87a549', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 34829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 34829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 34829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 34829}]}, 'public_addr': '172.21.15.136:6801/34829', 'cluster_addr': '172.21.15.136:6803/34829', 'heartbeat_back_addr': '172.21.15.136:6807/34829', 'heartbeat_front_addr': '172.21.15.136:6805/34829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3c57c228-dfaa-4e33-bded-27da0048d141', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 34830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 34830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6821', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 34830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6817', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 34830}]}, 'public_addr': '172.21.15.136:6811/34830', 'cluster_addr': '172.21.15.136:6815/34830', 'heartbeat_back_addr': '172.21.15.136:6823/34830', 'heartbeat_front_addr': '172.21.15.136:6819/34830', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '201f889f-dac1-4a26-a285-7a0ec21c3646', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 94, 'up_from': 132, 'up_thru': 92, 'down_at': 95, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6810', 'nonce': 48096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6814', 'nonce': 48096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6822', 'nonce': 48096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6818', 'nonce': 48096}]}, 'public_addr': '172.21.15.136:6810/48096', 'cluster_addr': '172.21.15.136:6814/48096', 'heartbeat_back_addr': '172.21.15.136:6822/48096', 'heartbeat_front_addr': '172.21.15.136:6818/48096', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '79ffd413-a9d3-4180-9691-d834d89c3f2e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 34874}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 34874}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 34874}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 34874}]}, 'public_addr': '172.21.15.136:6825/34874', 'cluster_addr': '172.21.15.136:6827/34874', 'heartbeat_back_addr': '172.21.15.136:6831/34874', 'heartbeat_front_addr': '172.21.15.136:6829/34874', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:49.068217+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:49.114373+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:50.647382+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:49.296499+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:50.239164+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 65536, 'last_purged_snaps_scrub': '2021-01-15T19:09:50.384906+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 65536, 'last_purged_snaps_scrub': '2021-01-15T19:10:00.995180+0000', 'dead_epoch': 95}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:50.678303+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'isaprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '2', 'm': '1', 'name': 'isaprofile', 'plugin': 'isa', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}, {'begin': 42, 'length': 1}, {'begin': 47, 'length': 1}, {'begin': 56, 'length': 1}, {'begin': 58, 'length': 1}, {'begin': 62, 'length': 1}, {'begin': 65, 'length': 1}, {'begin': 70, 'length': 1}, {'begin': 76, 'length': 1}, {'begin': 80, 'length': 1}, {'begin': 83, 'length': 2}, {'begin': 87, 'length': 3}, {'begin': 92, 'length': 2}, {'begin': 98, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2021-01-15T19:10:08.907 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2021-01-15T19:10:09.005 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:09.004+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:09.032 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:09.032 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:09.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:09.105 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:09.105+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:09.204 INFO:tasks.rados.rados.0.smithi001.stdout:565: done (5 left) 2021-01-15T19:10:09.205 INFO:tasks.rados.rados.0.smithi001.stdout:566: done (4 left) 2021-01-15T19:10:09.205 INFO:tasks.rados.rados.0.smithi001.stdout:567: done (3 left) 2021-01-15T19:10:09.205 INFO:tasks.rados.rados.0.smithi001.stdout:568: done (2 left) 2021-01-15T19:10:09.205 INFO:tasks.rados.rados.0.smithi001.stdout:569: done (1 left) 2021-01-15T19:10:09.206 INFO:tasks.rados.rados.0.smithi001.stdout:570: done (0 left) 2021-01-15T19:10:09.206 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 46 current snap is 63 2021-01-15T19:10:09.206 INFO:tasks.rados.rados.0.smithi001.stdout:571: seq_num 204 ranges {761856=622592,1384448=401408} 2021-01-15T19:10:09.207 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:09.207 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:09.208 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:09.206+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:09.212 INFO:tasks.rados.rados.0.smithi001.stdout:571: writing smithi00140323-46 from 761856 to 1384448 tid 1 2021-01-15T19:10:09.214 INFO:tasks.rados.rados.0.smithi001.stdout:571: writing smithi00140323-46 from 1384448 to 1785856 tid 2 2021-01-15T19:10:09.214 INFO:tasks.rados.rados.0.smithi001.stdout:572: delete oid 30 current snap is 63 2021-01-15T19:10:09.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:09.222 INFO:tasks.rados.rados.0.smithi001.stdout:572: done (1 left) 2021-01-15T19:10:09.222 INFO:tasks.rados.rados.0.smithi001.stdout:573: copy_from oid 41 from oid 11 current snap is 63 2021-01-15T19:10:09.222 INFO:tasks.rados.rados.0.smithi001.stdout:574: snap_create 2021-01-15T19:10:09.223 INFO:tasks.rados.rados.0.smithi001.stdout:571: finishing write tid 1 to smithi00140323-46 2021-01-15T19:10:09.223 INFO:tasks.rados.rados.0.smithi001.stdout:571: finishing write tid 2 to smithi00140323-46 2021-01-15T19:10:09.224 INFO:tasks.rados.rados.0.smithi001.stdout:571: finishing write tid 3 to smithi00140323-46 2021-01-15T19:10:09.226 INFO:tasks.rados.rados.0.smithi001.stdout:571: finishing write tid 4 to smithi00140323-46 2021-01-15T19:10:09.226 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 46 v 68 (ObjNum 204 snap 63 seq_num 204) dirty exists 2021-01-15T19:10:09.227 INFO:tasks.rados.rados.0.smithi001.stdout:571: left oid 46 (ObjNum 204 snap 63 seq_num 204) 2021-01-15T19:10:09.235 INFO:tasks.rados.rados.0.smithi001.stdout:573: finishing copy_from to smithi00140323-41 2021-01-15T19:10:09.235 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 41 v 52 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:10:09.238 INFO:tasks.rados.rados.0.smithi001.stdout:573: finishing copy_from racing read to smithi00140323-41 2021-01-15T19:10:09.308 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:09.307+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:09.345 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:09.345 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:09.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:09.439 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:09.438+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:09.510 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:09.509+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:09.520 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:09.520 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:09.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:09.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:10:09.614 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:09.613+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:09.694 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:09.695 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:09.704 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:09.712 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:09.711+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:09.813 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:09.812+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:09.868 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:09.869 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:09.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:09.919 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:09.918+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:10.000 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:10.016 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:10.015+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:10.042 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:10.043 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:10.052 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:10.117 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:10.116+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:10.147 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:10.167 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:10.187 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:10.204 INFO:tasks.rados.rados.0.smithi001.stdout:571: done (2 left) 2021-01-15T19:10:10.204 INFO:tasks.rados.rados.0.smithi001.stdout:573: done (1 left) 2021-01-15T19:10:10.204 INFO:tasks.rados.rados.0.smithi001.stdout:574: done (0 left) 2021-01-15T19:10:10.205 INFO:tasks.rados.rados.0.smithi001.stdout:575: setattr oid 14 current snap is 64 2021-01-15T19:10:10.206 INFO:tasks.rados.rados.0.smithi001.stdout:576: copy_from oid 11 from oid 9 current snap is 64 2021-01-15T19:10:10.206 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 32 current snap is 64 2021-01-15T19:10:10.206 INFO:tasks.rados.rados.0.smithi001.stdout:577: seq_num 206 ranges {2646016=655360,3301376=401408} 2021-01-15T19:10:10.207 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:10:10.211 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:10.218 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:10.217+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:10.222 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:10.222 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:10.224 INFO:tasks.rados.rados.0.smithi001.stdout:577: writing smithi00140323-32 from 2646016 to 3301376 tid 1 2021-01-15T19:10:10.227 INFO:tasks.rados.rados.0.smithi001.stdout:577: writing smithi00140323-32 from 3301376 to 3702784 tid 2 2021-01-15T19:10:10.228 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 93 (ObjNum 201 snap 62 seq_num 201) dirty exists 2021-01-15T19:10:10.228 INFO:tasks.rados.rados.0.smithi001.stdout:575: done (2 left) 2021-01-15T19:10:10.228 INFO:tasks.rados.rados.0.smithi001.stdout:578: copy_from oid 35 from oid 46 current snap is 64 2021-01-15T19:10:10.229 INFO:tasks.rados.rados.0.smithi001.stdout:579: snap_create 2021-01-15T19:10:10.232 INFO:tasks.rados.rados.0.smithi001.stdout:576: finishing copy_from to smithi00140323-11 2021-01-15T19:10:10.233 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 50 (ObjNum 139 snap 34 seq_num 139) dirty exists 2021-01-15T19:10:10.233 INFO:tasks.rados.rados.0.smithi001.stdout:576: finishing copy_from racing read to smithi00140323-11 2021-01-15T19:10:10.235 INFO:tasks.rados.rados.0.smithi001.stdout:577: finishing write tid 1 to smithi00140323-32 2021-01-15T19:10:10.236 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:10.237 INFO:tasks.rados.rados.0.smithi001.stdout:577: finishing write tid 2 to smithi00140323-32 2021-01-15T19:10:10.237 INFO:tasks.rados.rados.0.smithi001.stdout:577: finishing write tid 3 to smithi00140323-32 2021-01-15T19:10:10.238 INFO:tasks.rados.rados.0.smithi001.stdout:577: finishing write tid 4 to smithi00140323-32 2021-01-15T19:10:10.238 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 130 (ObjNum 206 snap 64 seq_num 206) dirty exists 2021-01-15T19:10:10.238 INFO:tasks.rados.rados.0.smithi001.stdout:577: left oid 32 (ObjNum 206 snap 64 seq_num 206) 2021-01-15T19:10:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:10.249 INFO:tasks.rados.rados.0.smithi001.stdout:578: finishing copy_from to smithi00140323-35 2021-01-15T19:10:10.250 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 68 (ObjNum 204 snap 63 seq_num 204) dirty exists 2021-01-15T19:10:10.250 INFO:tasks.rados.rados.0.smithi001.stdout:578: finishing copy_from racing read to smithi00140323-35 2021-01-15T19:10:10.268 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:10.290 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:10.309 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:10.310 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.310 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.311 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.312 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:10.320 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:10.319+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:10.406 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:10.406 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:10.406 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:10.406 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:10.415 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:10.420 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:10.419+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:10.521 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:10.520+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:10.579 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:10.580 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:10.580 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:10.580 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:10.580 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:10.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:10.621 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:10.621+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:10.722 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:10.722+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:10.753 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:10.754 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:10.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:10.824 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:10.823+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:10.898 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:10.898 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:10.898 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:10.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:10.951 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:10.950+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:11.026 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:11.025+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:11.039 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:11.040 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:11.049 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:11.133 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:11.132+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:11.205 INFO:tasks.rados.rados.0.smithi001.stdout:576: done (3 left) 2021-01-15T19:10:11.206 INFO:tasks.rados.rados.0.smithi001.stdout:577: done (2 left) 2021-01-15T19:10:11.206 INFO:tasks.rados.rados.0.smithi001.stdout:578: done (1 left) 2021-01-15T19:10:11.206 INFO:tasks.rados.rados.0.smithi001.stdout:579: done (0 left) 2021-01-15T19:10:11.206 INFO:tasks.rados.rados.0.smithi001.stdout:580: copy_from oid 8 from oid 43 current snap is 65 2021-01-15T19:10:11.209 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 38 current snap is 65 2021-01-15T19:10:11.209 INFO:tasks.rados.rados.0.smithi001.stdout:581: seq_num 207 ranges {4456448=671744,5128192=401408} 2021-01-15T19:10:11.227 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:11.227 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:11.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:11.228 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:11.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.228 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:11.228 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:11.229 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.970934658999994, 2021-01-15T19:10:11.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:11.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:11.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.231 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:11.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:11.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:11.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:11.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:11.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:11.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:11.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:11.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:11.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:11.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:11.244 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.970769546, 2021-01-15T19:10:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:11.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:11.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:11.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:11.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:11.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:11.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:11.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:11.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:11.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:11.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:11.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:11.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:11.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:11.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:11.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:11.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:11.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:11.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:11.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:11.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:11.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.260 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:11.261 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:11.261 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 71.963489496999998, 2021-01-15T19:10:11.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:11.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.262 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:11.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.263 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:11.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:11.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:11.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:11.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:11.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:11.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:11.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:11.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:11.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:11.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:11.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:11.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:11.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:11.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:11.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:11.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:11.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:11.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:11.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:11.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:11.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:11.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:11.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.275 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.275 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.276 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:11.276 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:11.276 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 71.963362864000004, 2021-01-15T19:10:11.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:11.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.277 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:11.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.278 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:11.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:11.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:11.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:11.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:11.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:11.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:11.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:11.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:11.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:11.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:11.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:11.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:11.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:11.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:11.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:11.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:11.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:11.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:11.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:11.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:11.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:11.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:11.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:11.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:11.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:11.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.292 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:11.293 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:11.293 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.880625899000002, 2021-01-15T19:10:11.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:11.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.294 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.294 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.294 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:11.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.295 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:11.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:11.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:11.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:11.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:11.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:11.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:11.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:11.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:11.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:11.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:11.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:11.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:11.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:11.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:11.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:11.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:11.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:11.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:11.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:11.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:11.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.308 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:11.309 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:11.309 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.880476285, 2021-01-15T19:10:11.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:11.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.310 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:11.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.311 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:11.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:11.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:11.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:11.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:11.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:11.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:11.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:11.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:11.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.860178603000001, 2021-01-15T19:10:11.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.326 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:11.342 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.857121952, 2021-01-15T19:10:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:11.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.854108689999997, 2021-01-15T19:10:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.360 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.361 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:11.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:11.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:11.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:11.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:11.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:11.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:11.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:11.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:11.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:11.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:11.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:11.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:11.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.376 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.851361079, 2021-01-15T19:10:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.377 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:11.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:11.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:11.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:11.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:11.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:11.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:11.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:11.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:11.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:11.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:11.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:11.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:11.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:11.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.850927257000002, 2021-01-15T19:10:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:11.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.850803098, 2021-01-15T19:10:11.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.412 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:11.427 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.746126562999997, 2021-01-15T19:10:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.428 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:11.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:11.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:11.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:11.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:11.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:11.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:11.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:11.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:11.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:11.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:11.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:11.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:11.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:11.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:11.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:11.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:11.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:11.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:11.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:11.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:11.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:11.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:11.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:11.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:11.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:11.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:11.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:11.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:11.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:11.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:11.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:11.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.447 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:11.448 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.865905484000002, 2021-01-15T19:10:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.449 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:11.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:11.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:11.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:11.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:11.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:11.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:11.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:11.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:11.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:11.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:11.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:11.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:11.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:11.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:11.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:11.462 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.463 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.463 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.463 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:11.464 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:11.464 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.836187326999998, 2021-01-15T19:10:11.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:11.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:11.465 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:11.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:11.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:11.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:11.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:11.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:11.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:11.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:11.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:11.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:11.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:11.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:11.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:11.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:11.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:11.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.473 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.474 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:11.474 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:11.474 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.832101160000001, 2021-01-15T19:10:11.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:11.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:11.475 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:11.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:11.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:11.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:11.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:11.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:11.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:11.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:11.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:11.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:11.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:11.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:11.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:11.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:11.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:11.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.483 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.484 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:11.484 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:11.485 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 57.831965408999999, 2021-01-15T19:10:11.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:11.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:11.485 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:11.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:11.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:11.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:11.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:11.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:11.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:11.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:11.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:11.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:11.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:11.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:11.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:11.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:11.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:11.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.494 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:11.495 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:11.495 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.993466426000001, 2021-01-15T19:10:11.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:11.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:11.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.497 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:11.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:11.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:11.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:11.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:11.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:11.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:11.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:11.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:11.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:11.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:11.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:11.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:11.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:11.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:11.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:11.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:11.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:11.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:11.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:11.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:11.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.508 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.509 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:11.509 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:11.509 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.0188869620000001, 2021-01-15T19:10:11.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:11.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:11.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:11.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.511 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:11.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:11.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:11.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:11.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:11.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:11.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:11.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:11.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:11.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:11.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:11.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:11.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:11.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:11.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:11.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:11.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:11.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:11.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:11.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:11.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:11.525 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.525 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.525 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.526 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:11.526 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:11.526 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.0187553009999999, 2021-01-15T19:10:11.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:11.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:11.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:11.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:11.527 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:11.528 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:11.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:11.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.528 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:11.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:11.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:11.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:11.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:11.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:11.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:11.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:11.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:11.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:11.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:11.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:11.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:11.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:11.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:11.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:11.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:11.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:11.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:11.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:11.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:11.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:11.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:11.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:11.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:11.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:11.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:11.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:11.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:11.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:11.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:11.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:11.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:11.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:11.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:11.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:11.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:11.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:11.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:11.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:11.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:11.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:11.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:11.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:11.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:11.543 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.544 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:11.544 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:11.544 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:11.545 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:11.225+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:11.546 INFO:tasks.rados.rados.0.smithi001.stdout:581: writing smithi00140323-38 from 4456448 to 5128192 tid 1 2021-01-15T19:10:11.547 INFO:tasks.rados.rados.0.smithi001.stdout:581: writing smithi00140323-38 from 5128192 to 5529600 tid 2 2021-01-15T19:10:11.547 INFO:tasks.rados.rados.0.smithi001.stdout:582: snap_create 2021-01-15T19:10:11.547 INFO:tasks.rados.rados.0.smithi001.stdout:581: finishing write tid 1 to smithi00140323-38 2021-01-15T19:10:11.547 INFO:tasks.rados.rados.0.smithi001.stdout:580: finishing copy_from to smithi00140323-8 2021-01-15T19:10:11.548 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 34 (ObjNum 127 snap 30 seq_num 127) dirty exists 2021-01-15T19:10:11.548 INFO:tasks.rados.rados.0.smithi001.stdout:580: finishing copy_from racing read to smithi00140323-8 2021-01-15T19:10:11.548 INFO:tasks.rados.rados.0.smithi001.stdout:581: finishing write tid 2 to smithi00140323-38 2021-01-15T19:10:11.548 INFO:tasks.rados.rados.0.smithi001.stdout:581: finishing write tid 3 to smithi00140323-38 2021-01-15T19:10:11.548 INFO:tasks.rados.rados.0.smithi001.stdout:581: finishing write tid 4 to smithi00140323-38 2021-01-15T19:10:11.549 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 61 (ObjNum 207 snap 65 seq_num 207) dirty exists 2021-01-15T19:10:11.549 INFO:tasks.rados.rados.0.smithi001.stdout:581: left oid 38 (ObjNum 207 snap 65 seq_num 207) 2021-01-15T19:10:11.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:10:11.550 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:11.545+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:11.647 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:11.646+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:11.723 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:11.724 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:11.724 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:11.733 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:10:11.777 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:11.776+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:11.848 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:11.847+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:11.864 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:11.864 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:11.865 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:11.865 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:11.865 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:11.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:10:11.949 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:11.949+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:12.049 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:12.049 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:12.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:12.050 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:12.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:10:12.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:12.051 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 141.52983181799999, 2021-01-15T19:10:12.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:10:12.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:10:12.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:10:12.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:10:12.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.054 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:12.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:12.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:10:12.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:10:12.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:10:12.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:10:12.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:10:12.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:10:12.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:10:12.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:10:12.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:10:12.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:10:12.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:10:12.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:12.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:10:12.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:10:12.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:10:12.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:10:12.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:10:12.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:10:12.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:829 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:12.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:12.066 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.864117497999999, 2021-01-15T19:10:12.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001, 2021-01-15T19:10:12.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 829 2021-01-15T19:10:12.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.068 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:12.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:12.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4360000000000001e-05 2021-01-15T19:10:12.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:12.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0169999999999999e-06 2021-01-15T19:10:12.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182630+0000", 2021-01-15T19:10:12.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4198e-05 2021-01-15T19:10:12.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182644+0000", 2021-01-15T19:10:12.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1253999999999999e-05 2021-01-15T19:10:12.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182656+0000", 2021-01-15T19:10:12.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9756999999999999e-05 2021-01-15T19:10:12.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182695+0000", 2021-01-15T19:10:12.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2819999999999996e-06 2021-01-15T19:10:12.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:12.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182704+0000", 2021-01-15T19:10:12.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048051730000000003 2021-01-15T19:10:12.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187509+0000", 2021-01-15T19:10:12.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114002 2021-01-15T19:10:12.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187623+0000", 2021-01-15T19:10:12.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012123558 2021-01-15T19:10:12.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.199746+0000", 2021-01-15T19:10:12.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001 2021-01-15T19:10:12.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.083 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=315, reqid=client.4418.0:833, at_version=80'84, trim_to=0'0, roll_forward_to=78'82))", 2021-01-15T19:10:12.083 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:12.084 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.859555530999998, 2021-01-15T19:10:12.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001, 2021-01-15T19:10:12.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:12.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:12.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6477000000000002e-05 2021-01-15T19:10:12.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187187+0000", 2021-01-15T19:10:12.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4749e-05 2021-01-15T19:10:12.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187212+0000", 2021-01-15T19:10:12.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5977e-05 2021-01-15T19:10:12.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187228+0000", 2021-01-15T19:10:12.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0770000000000008e-06 2021-01-15T19:10:12.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187236+0000", 2021-01-15T19:10:12.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014010800000000001 2021-01-15T19:10:12.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187376+0000", 2021-01-15T19:10:12.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8624999999999999e-05 2021-01-15T19:10:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187395+0000", 2021-01-15T19:10:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013611689 2021-01-15T19:10:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201006+0000", 2021-01-15T19:10:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7980999999999999e-05 2021-01-15T19:10:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201064+0000", 2021-01-15T19:10:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001 2021-01-15T19:10:12.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.099 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:971 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.837412137000001, 2021-01-15T19:10:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002, 2021-01-15T19:10:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 971 2021-01-15T19:10:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2442999999999998e-05 2021-01-15T19:10:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209337+0000", 2021-01-15T19:10:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0950000000000002e-06 2021-01-15T19:10:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209346+0000", 2021-01-15T19:10:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9199999999999999e-05 2021-01-15T19:10:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209365+0000", 2021-01-15T19:10:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5750000000000007e-06 2021-01-15T19:10:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209374+0000", 2021-01-15T19:10:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074262999999999998 2021-01-15T19:10:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210117+0000", 2021-01-15T19:10:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022372900000000001 2021-01-15T19:10:12.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210341+0000", 2021-01-15T19:10:12.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016740338 2021-01-15T19:10:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227081+0000", 2021-01-15T19:10:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002 2021-01-15T19:10:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:972 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.837265381000002, 2021-01-15T19:10:12.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999, 2021-01-15T19:10:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.114 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 972 2021-01-15T19:10:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5875000000000003e-05 2021-01-15T19:10:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209477+0000", 2021-01-15T19:10:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7499999999999992e-06 2021-01-15T19:10:12.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209486+0000", 2021-01-15T19:10:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4825e-05 2021-01-15T19:10:12.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209500+0000", 2021-01-15T19:10:12.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209508+0000", 2021-01-15T19:10:12.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012352109999999999 2021-01-15T19:10:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210743+0000", 2021-01-15T19:10:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012467700000000001 2021-01-15T19:10:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210867+0000", 2021-01-15T19:10:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018560825 2021-01-15T19:10:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.229428+0000", 2021-01-15T19:10:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999 2021-01-15T19:10:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:973 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:12.126 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 48.837134048999999, 2021-01-15T19:10:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001, 2021-01-15T19:10:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 973 2021-01-15T19:10:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:12.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7021999999999997e-05 2021-01-15T19:10:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209609+0000", 2021-01-15T19:10:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3140000000000004e-06 2021-01-15T19:10:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209618+0000", 2021-01-15T19:10:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4259e-05 2021-01-15T19:10:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209632+0000", 2021-01-15T19:10:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2021-01-15T19:10:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209639+0000", 2021-01-15T19:10:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039100000000000003 2021-01-15T19:10:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213549+0000", 2021-01-15T19:10:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124948 2021-01-15T19:10:12.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213674+0000", 2021-01-15T19:10:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018190644999999998 2021-01-15T19:10:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.231864+0000", 2021-01-15T19:10:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001 2021-01-15T19:10:12.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1093 3.10s0 3:095f0b32:::smithi00140323-47:head [rollback 50] snapc 50=[50,4b,44,41,26] ondisk+write+known_if_redirected e109)", 2021-01-15T19:10:12.141 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.810459915999999, 2021-01-15T19:10:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242, 2021-01-15T19:10:12.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1093 2021-01-15T19:10:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:12.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3699999999999998e-05 2021-01-15T19:10:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236270+0000", 2021-01-15T19:10:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4700000000000008e-06 2021-01-15T19:10:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236280+0000", 2021-01-15T19:10:12.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6716999999999999e-05 2021-01-15T19:10:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236296+0000", 2021-01-15T19:10:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5490000000000006e-06 2021-01-15T19:10:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236305+0000", 2021-01-15T19:10:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4991999999999998e-05 2021-01-15T19:10:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236360+0000", 2021-01-15T19:10:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3660000000000003e-06 2021-01-15T19:10:12.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:12.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236367+0000", 2021-01-15T19:10:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005477549 2021-01-15T19:10:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241845+0000", 2021-01-15T19:10:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137458 2021-01-15T19:10:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241982+0000", 2021-01-15T19:10:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097224410000000001 2021-01-15T19:10:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.251705+0000", 2021-01-15T19:10:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242 2021-01-15T19:10:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.159 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 109/95 ECSubWrite(tid=263, reqid=client.4418.0:1090, at_version=109'60, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:12.160 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.806644278, 2021-01-15T19:10:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999, 2021-01-15T19:10:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.161 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:12.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8936e-05 2021-01-15T19:10:12.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240091+0000", 2021-01-15T19:10:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1778000000000001e-05 2021-01-15T19:10:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240113+0000", 2021-01-15T19:10:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6714999999999998e-05 2021-01-15T19:10:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240129+0000", 2021-01-15T19:10:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240137+0000", 2021-01-15T19:10:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9104999999999998e-05 2021-01-15T19:10:12.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240186+0000", 2021-01-15T19:10:12.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240194+0000", 2021-01-15T19:10:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017479262999999998 2021-01-15T19:10:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257674+0000", 2021-01-15T19:10:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9951999999999999e-05 2021-01-15T19:10:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257724+0000", 2021-01-15T19:10:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999 2021-01-15T19:10:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 110/95 ECSubWrite(tid=269, reqid=client.4418.0:1098, at_version=110'64, trim_to=0'0, roll_forward_to=109'62))", 2021-01-15T19:10:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.852583267999997, 2021-01-15T19:10:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999, 2021-01-15T19:10:12.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5725e-05 2021-01-15T19:10:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194149+0000", 2021-01-15T19:10:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3603000000000003e-05 2021-01-15T19:10:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194182+0000", 2021-01-15T19:10:12.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0721e-05 2021-01-15T19:10:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194203+0000", 2021-01-15T19:10:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3929999999999997e-06 2021-01-15T19:10:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194212+0000", 2021-01-15T19:10:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0593000000000006e-05 2021-01-15T19:10:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194293+0000", 2021-01-15T19:10:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3596e-05 2021-01-15T19:10:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194307+0000", 2021-01-15T19:10:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017420983000000001 2021-01-15T19:10:12.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211728+0000", 2021-01-15T19:10:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6654999999999999e-05 2021-01-15T19:10:12.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211784+0000", 2021-01-15T19:10:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999 2021-01-15T19:10:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.191 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1116 3.4s1 3:21d37027:::smithi00140323-27:head [copy-from ver 54] snapc 57=[57,56,53,50,4b,44,41,26] ondisk+write+known_if_redirected e114)", 2021-01-15T19:10:12.192 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:12.192 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.794144666000001, 2021-01-15T19:10:12.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999, 2021-01-15T19:10:12.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:12.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.194 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1116 2021-01-15T19:10:12.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.195 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:12.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:12.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3304000000000002e-05 2021-01-15T19:10:12.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252605+0000", 2021-01-15T19:10:12.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0723000000000001e-05 2021-01-15T19:10:12.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252616+0000", 2021-01-15T19:10:12.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0720000000000002e-05 2021-01-15T19:10:12.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252636+0000", 2021-01-15T19:10:12.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1799e-05 2021-01-15T19:10:12.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252648+0000", 2021-01-15T19:10:12.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5387000000000003e-05 2021-01-15T19:10:12.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252714+0000", 2021-01-15T19:10:12.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029657199999999999 2021-01-15T19:10:12.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253010+0000", 2021-01-15T19:10:12.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030464359 2021-01-15T19:10:12.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.283475+0000", 2021-01-15T19:10:12.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011930975999999999 2021-01-15T19:10:12.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295405+0000", 2021-01-15T19:10:12.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2021-01-15T19:10:12.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:12.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295488+0000", 2021-01-15T19:10:12.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074145999999999999 2021-01-15T19:10:12.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296229+0000", 2021-01-15T19:10:12.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999 2021-01-15T19:10:12.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.210 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.210 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1117 3.4s1 3:21d37027:::smithi00140323-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e114)", 2021-01-15T19:10:12.211 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:12.211 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.793980170000001, 2021-01-15T19:10:12.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003, 2021-01-15T19:10:12.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.212 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1117 2021-01-15T19:10:12.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.213 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:12.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3238000000000002e-05 2021-01-15T19:10:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252749+0000", 2021-01-15T19:10:12.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8820000000000001e-06 2021-01-15T19:10:12.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252757+0000", 2021-01-15T19:10:12.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6419e-05 2021-01-15T19:10:12.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252774+0000", 2021-01-15T19:10:12.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9879999999999998e-06 2021-01-15T19:10:12.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252781+0000", 2021-01-15T19:10:12.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051378399999999999 2021-01-15T19:10:12.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253294+0000", 2021-01-15T19:10:12.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8572000000000006e-05 2021-01-15T19:10:12.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:12.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253373+0000", 2021-01-15T19:10:12.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040965935000000002 2021-01-15T19:10:12.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294339+0000", 2021-01-15T19:10:12.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8033000000000001e-05 2021-01-15T19:10:12.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:12.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294437+0000", 2021-01-15T19:10:12.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018291189999999999 2021-01-15T19:10:12.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296266+0000", 2021-01-15T19:10:12.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0880999999999999e-05 2021-01-15T19:10:12.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296357+0000", 2021-01-15T19:10:12.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016417 2021-01-15T19:10:12.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296521+0000", 2021-01-15T19:10:12.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003 2021-01-15T19:10:12.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.228 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.228 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s0 3.18aae252 (undecoded) ondisk+write+peerstat_old+known_if_redirected e119)", 2021-01-15T19:10:12.229 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:12.229 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.767343836999999, 2021-01-15T19:10:12.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263, 2021-01-15T19:10:12.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:10:12.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.231 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:12.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:12.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5269000000000003e-05 2021-01-15T19:10:12.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279388+0000", 2021-01-15T19:10:12.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4849999999999998e-06 2021-01-15T19:10:12.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279397+0000", 2021-01-15T19:10:12.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.63e-05 2021-01-15T19:10:12.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279414+0000", 2021-01-15T19:10:12.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:12.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279422+0000", 2021-01-15T19:10:12.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5129e-05 2021-01-15T19:10:12.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279477+0000", 2021-01-15T19:10:12.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1559999999999997e-06 2021-01-15T19:10:12.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:12.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279481+0000", 2021-01-15T19:10:12.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.92165986799999999 2021-01-15T19:10:12.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201141+0000", 2021-01-15T19:10:12.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9066999999999999e-05 2021-01-15T19:10:12.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201160+0000", 2021-01-15T19:10:12.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263 2021-01-15T19:10:12.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.243 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4166.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e99)", 2021-01-15T19:10:12.243 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:12.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.490974613999999, 2021-01-15T19:10:12.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116, 2021-01-15T19:10:12.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4166", 2021-01-15T19:10:12.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4191424063", 2021-01-15T19:10:12.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4 2021-01-15T19:10:12.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:12.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:12.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7705999999999999e-05 2021-01-15T19:10:12.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555739+0000", 2021-01-15T19:10:12.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8659999999999995e-06 2021-01-15T19:10:12.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555748+0000", 2021-01-15T19:10:12.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2021-01-15T19:10:12.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555761+0000", 2021-01-15T19:10:12.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9859999999999994e-06 2021-01-15T19:10:12.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555769+0000", 2021-01-15T19:10:12.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034585600000000002 2021-01-15T19:10:12.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556115+0000", 2021-01-15T19:10:12.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:12.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:12.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556129+0000", 2021-01-15T19:10:12.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.65062051300000001 2021-01-15T19:10:12.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206749+0000", 2021-01-15T19:10:12.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014163399999999999 2021-01-15T19:10:12.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206891+0000", 2021-01-15T19:10:12.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116 2021-01-15T19:10:12.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.259 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.4s1 124/122 ECSubWrite(tid=344, reqid=client.4418.0:1180, at_version=124'63, trim_to=0'0, roll_forward_to=123'61))", 2021-01-15T19:10:12.259 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:12.259 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.814474251, 2021-01-15T19:10:12.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011987622999999999, 2021-01-15T19:10:12.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:12.260 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:12.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232212+0000", 2021-01-15T19:10:12.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6241e-05 2021-01-15T19:10:12.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232258+0000", 2021-01-15T19:10:12.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:12.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232273+0000", 2021-01-15T19:10:12.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4020000000000002e-06 2021-01-15T19:10:12.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232281+0000", 2021-01-15T19:10:12.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3479999999999997e-06 2021-01-15T19:10:12.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232285+0000", 2021-01-15T19:10:12.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2518999999999998e-05 2021-01-15T19:10:12.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232318+0000", 2021-01-15T19:10:12.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:12.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.232325+0000", 2021-01-15T19:10:12.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011821569000000001 2021-01-15T19:10:12.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.244147+0000", 2021-01-15T19:10:12.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3057999999999999e-05 2021-01-15T19:10:12.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.244200+0000", 2021-01-15T19:10:12.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011987622999999999 2021-01-15T19:10:12.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.271 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.271 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1187 3.12s0 3:4bacf681:::smithi00140323-17:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:12.272 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:12.272 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.820301269, 2021-01-15T19:10:12.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999, 2021-01-15T19:10:12.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:12.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1187 2021-01-15T19:10:12.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.274 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:12.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:12.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8656999999999998e-05 2021-01-15T19:10:12.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226444+0000", 2021-01-15T19:10:12.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:12.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226452+0000", 2021-01-15T19:10:12.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3689000000000001e-05 2021-01-15T19:10:12.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226466+0000", 2021-01-15T19:10:12.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8540000000000002e-06 2021-01-15T19:10:12.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226472+0000", 2021-01-15T19:10:12.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2799999999999997e-05 2021-01-15T19:10:12.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226514+0000", 2021-01-15T19:10:12.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1725999999999999e-05 2021-01-15T19:10:12.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:12.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226526+0000", 2021-01-15T19:10:12.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066831169999999997 2021-01-15T19:10:12.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.233209+0000", 2021-01-15T19:10:12.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011647090000000001 2021-01-15T19:10:12.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234374+0000", 2021-01-15T19:10:12.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00061833799999999996 2021-01-15T19:10:12.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234992+0000", 2021-01-15T19:10:12.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001433157 2021-01-15T19:10:12.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.236425+0000", 2021-01-15T19:10:12.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044118559999999996 2021-01-15T19:10:12.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:12.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240837+0000", 2021-01-15T19:10:12.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152409 2021-01-15T19:10:12.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240990+0000", 2021-01-15T19:10:12.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999 2021-01-15T19:10:12.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.289 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.290 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.290 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1188 3.12s0 3:4bacf681:::smithi00140323-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:12.291 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:12.291 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.820166008999999, 2021-01-15T19:10:12.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001, 2021-01-15T19:10:12.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1188 2021-01-15T19:10:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:12.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:12.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0703e-05 2021-01-15T19:10:12.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226561+0000", 2021-01-15T19:10:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5660000000000001e-06 2021-01-15T19:10:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226569+0000", 2021-01-15T19:10:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3385e-05 2021-01-15T19:10:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226582+0000", 2021-01-15T19:10:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226587+0000", 2021-01-15T19:10:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.409e-05 2021-01-15T19:10:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226632+0000", 2021-01-15T19:10:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1242e-05 2021-01-15T19:10:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226643+0000", 2021-01-15T19:10:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086436099999999995 2021-01-15T19:10:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235286+0000", 2021-01-15T19:10:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016845599999999999 2021-01-15T19:10:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235455+0000", 2021-01-15T19:10:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060319140000000002 2021-01-15T19:10:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241487+0000", 2021-01-15T19:10:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014580699999999999 2021-01-15T19:10:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241633+0000", 2021-01-15T19:10:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001527306 2021-01-15T19:10:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.243160+0000", 2021-01-15T19:10:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001 2021-01-15T19:10:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.310 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1192 3.12s0 3:4e4266df:::smithi00140323-37:head [append 0~557056] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:12.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.791061923999999, 2021-01-15T19:10:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012911791000000001, 2021-01-15T19:10:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1192 2021-01-15T19:10:12.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:12.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.255624+0000", 2021-01-15T19:10:12.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042538899999999998 2021-01-15T19:10:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256050+0000", 2021-01-15T19:10:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1664999999999999e-05 2021-01-15T19:10:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256061+0000", 2021-01-15T19:10:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0514e-05 2021-01-15T19:10:12.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256082+0000", 2021-01-15T19:10:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0865e-05 2021-01-15T19:10:12.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256093+0000", 2021-01-15T19:10:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7393999999999999e-05 2021-01-15T19:10:12.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256160+0000", 2021-01-15T19:10:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013985700000000001 2021-01-15T19:10:12.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256300+0000", 2021-01-15T19:10:12.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001360053 2021-01-15T19:10:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.257660+0000", 2021-01-15T19:10:12.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010289375999999999 2021-01-15T19:10:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267950+0000", 2021-01-15T19:10:12.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051317300000000001 2021-01-15T19:10:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:12.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268463+0000", 2021-01-15T19:10:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3504999999999995e-05 2021-01-15T19:10:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268536+0000", 2021-01-15T19:10:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012911791000000001 2021-01-15T19:10:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1193 3.12s0 3:4e4266df:::smithi00140323-37:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:12.328 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.790544657, 2021-01-15T19:10:12.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013858986, 2021-01-15T19:10:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1193 2021-01-15T19:10:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256142+0000", 2021-01-15T19:10:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7895999999999997e-05 2021-01-15T19:10:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256190+0000", 2021-01-15T19:10:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5399999999999998e-06 2021-01-15T19:10:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256197+0000", 2021-01-15T19:10:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.252e-05 2021-01-15T19:10:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256210+0000", 2021-01-15T19:10:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1340000000000001e-06 2021-01-15T19:10:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256216+0000", 2021-01-15T19:10:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010534175 2021-01-15T19:10:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.266750+0000", 2021-01-15T19:10:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109787 2021-01-15T19:10:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.266860+0000", 2021-01-15T19:10:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00057329899999999999 2021-01-15T19:10:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267433+0000", 2021-01-15T19:10:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076203700000000002 2021-01-15T19:10:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.268195+0000", 2021-01-15T19:10:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001719424 2021-01-15T19:10:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.269914+0000", 2021-01-15T19:10:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6174000000000005e-05 2021-01-15T19:10:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270001+0000", 2021-01-15T19:10:12.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013858986 2021-01-15T19:10:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1194 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.790423049999999, 2021-01-15T19:10:12.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019, 2021-01-15T19:10:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.348 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1194 2021-01-15T19:10:12.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8844999999999998e-05 2021-01-15T19:10:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256302+0000", 2021-01-15T19:10:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9030000000000004e-06 2021-01-15T19:10:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256310+0000", 2021-01-15T19:10:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4749999999999999e-05 2021-01-15T19:10:12.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256325+0000", 2021-01-15T19:10:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2370000000000003e-06 2021-01-15T19:10:12.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256332+0000", 2021-01-15T19:10:12.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011409671999999999 2021-01-15T19:10:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267742+0000", 2021-01-15T19:10:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017902200000000001 2021-01-15T19:10:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267921+0000", 2021-01-15T19:10:12.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027501040000000002 2021-01-15T19:10:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270671+0000", 2021-01-15T19:10:12.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020387700000000001 2021-01-15T19:10:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270875+0000", 2021-01-15T19:10:12.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012356089999999999 2021-01-15T19:10:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.272110+0000", 2021-01-15T19:10:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019 2021-01-15T19:10:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1331 3.10s0 3:0c50693c:::smithi00140323-16:head [append 0~507904] snapc 6e=[6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e139)", 2021-01-15T19:10:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:07.343952+0000", 2021-01-15T19:10:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.7027346000000003, 2021-01-15T19:10:12.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011999858, 2021-01-15T19:10:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:12.365 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1331 2021-01-15T19:10:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.343952+0000", 2021-01-15T19:10:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.343952+0000", 2021-01-15T19:10:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023507700000000001 2021-01-15T19:10:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.344187+0000", 2021-01-15T19:10:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2847000000000001e-05 2021-01-15T19:10:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.344200+0000", 2021-01-15T19:10:12.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5673000000000001e-05 2021-01-15T19:10:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.344215+0000", 2021-01-15T19:10:12.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5950000000000005e-06 2021-01-15T19:10:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.344225+0000", 2021-01-15T19:10:12.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6971000000000006e-05 2021-01-15T19:10:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.344292+0000", 2021-01-15T19:10:12.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5289999999999997e-06 2021-01-15T19:10:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.344301+0000", 2021-01-15T19:10:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0022530839999999998 2021-01-15T19:10:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:12.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.346554+0000", 2021-01-15T19:10:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017753199999999999 2021-01-15T19:10:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:12.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.346732+0000", 2021-01-15T19:10:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001047297 2021-01-15T19:10:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.347779+0000", 2021-01-15T19:10:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0077556220000000002 2021-01-15T19:10:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.355535+0000", 2021-01-15T19:10:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8746999999999995e-05 2021-01-15T19:10:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.355634+0000", 2021-01-15T19:10:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031788399999999999 2021-01-15T19:10:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:12.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:07.355952+0000", 2021-01-15T19:10:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011999858 2021-01-15T19:10:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.381 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:12.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:12.382 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:12.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:10:12.384 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:12.385 INFO:tasks.rados.rados.0.smithi001.stdout:580: done (2 left) 2021-01-15T19:10:12.385 INFO:tasks.rados.rados.0.smithi001.stdout:581: done (1 left) 2021-01-15T19:10:12.386 INFO:tasks.rados.rados.0.smithi001.stdout:582: done (0 left) 2021-01-15T19:10:12.386 INFO:tasks.rados.rados.0.smithi001.stdout:583: snap_remove snap 59 2021-01-15T19:10:12.386 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:12.383+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:12.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:12.485 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:12.484+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:12.564 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:12.564 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:12.565 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:12.579 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:12.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:12.592 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:12.590+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:12.613 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:12.634 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:12.655 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:12.676 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:12.687 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:12.686+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:12.697 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:12.711 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:12.712 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:12.718 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:12.722 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:12.740 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.742 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.743 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.743 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.743 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.743 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:12.793 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:12.792+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:12.871 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:12.871 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:12.881 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:12.888 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:12.887+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:12.989 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:12.987+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:13.053 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:13.054 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:13.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:13.089 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:13.089+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:13.190 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:13.189+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:13.212 INFO:tasks.rados.rados.0.smithi001.stdout:583: done (0 left) 2021-01-15T19:10:13.212 INFO:tasks.rados.rados.0.smithi001.stdout:584: read oid 26 snap 55 2021-01-15T19:10:13.213 INFO:tasks.rados.rados.0.smithi001.stdout:584: expect (ObjNum 103 snap 17 seq_num 103) 2021-01-15T19:10:13.213 INFO:tasks.rados.rados.0.smithi001.stdout:585: delete oid 12 current snap is 66 2021-01-15T19:10:13.222 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:10:13.232 INFO:tasks.rados.rados.0.smithi001.stdout:585: done (1 left) 2021-01-15T19:10:13.233 INFO:tasks.rados.rados.0.smithi001.stdout:586: snap_remove snap 23 2021-01-15T19:10:13.252 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:13.252 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:13.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:13.290 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:13.290+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:13.391 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:13.391+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:13.434 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:13.434 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:13.444 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:13.493 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:13.492+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:13.579 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:13.579 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:13.590 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:13.594 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:13.593+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:13.695 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:13.694+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:13.714 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:13.714 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:13.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:13.808 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:13.807+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:13.889 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:13.890 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:13.896 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:13.895+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:13.899 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:13.909 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:13.909 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:10:13.910 INFO:tasks.thrashosds.thrasher:fixing pg num pool rbd 2021-01-15T19:10:13.911 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:10:13.997 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:13.995+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:14.064 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:14.065 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:14.074 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:14.098 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:14.097+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:14.199 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:14.199 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:14.200 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:14.198+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:14.209 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:14.220 INFO:tasks.rados.rados.0.smithi001.stdout:586: done (1 left) 2021-01-15T19:10:14.220 INFO:tasks.rados.rados.0.smithi001.stdout:587: snap_remove snap 38 2021-01-15T19:10:14.238 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:10:14.244 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:10:14.257 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2021-01-15T19:10:14.214115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206094336,"num_objects":169,"num_object_clones":121,"num_object_copies":505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":10,"num_read":457,"num_read_kb":426971,"num_write":743,"num_write_kb":173441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":248659968,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2565,"ondisk_log_size":2565,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":479960,"kb_used_data":309096,"kb_used_omap":0,"kb_used_meta":170816,"kb_avail":565751080,"statfs":{"total":579820584960,"available":579329105920,"internally_reserved":0,"allocated":316514304,"data_stored":559080822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":174915584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21364736,"num_objects":18,"num_object_clones":18,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":27,"num_read_kb":16246,"num_write":55,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013015"},"pg_stats":[{"pgid":"3.19","version":"102'55","reported_seq":"237","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215471+0000","last_change":"2021-01-15T19:09:27.212298+0000","last_active":"2021-01-15T19:10:09.215471+0000","last_peered":"2021-01-15T19:10:09.215471+0000","last_clean":"2021-01-15T19:10:09.215471+0000","last_became_active":"2021-01-15T19:08:46.196966+0000","last_became_peered":"2021-01-15T19:08:46.196966+0000","last_unstale":"2021-01-15T19:10:09.215471+0000","last_undegraded":"2021-01-15T19:10:09.215471+0000","last_fullsized":"2021-01-15T19:10:09.215471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'51","last_scrub_stamp":"2021-01-15T19:09:27.212108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:27.212108+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":18,"stat_sum":{"num_bytes":5300224,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"141'68","reported_seq":"361","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.203238+0000","last_change":"2021-01-15T19:10:05.333388+0000","last_active":"2021-01-15T19:10:10.203238+0000","last_peered":"2021-01-15T19:10:10.203238+0000","last_clean":"2021-01-15T19:10:10.203238+0000","last_became_active":"2021-01-15T19:09:51.207469+0000","last_became_peered":"2021-01-15T19:09:51.207469+0000","last_unstale":"2021-01-15T19:10:10.203238+0000","last_undegraded":"2021-01-15T19:10:10.203238+0000","last_fullsized":"2021-01-15T19:10:10.203238+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"51'34","last_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_deep_scrub":"51'34","last_deep_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.694318+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2547712,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":2235,"num_write":13,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":3047424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"3.17","version":"50'30","reported_seq":"158","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.208166+0000","last_change":"2021-01-15T19:10:05.850117+0000","last_active":"2021-01-15T19:10:10.208166+0000","last_peered":"2021-01-15T19:10:10.208166+0000","last_clean":"2021-01-15T19:10:10.208166+0000","last_became_active":"2021-01-15T19:09:21.207252+0000","last_became_peered":"2021-01-15T19:09:21.207252+0000","last_unstale":"2021-01-15T19:10:10.208166+0000","last_undegraded":"2021-01-15T19:10:10.208166+0000","last_fullsized":"2021-01-15T19:10:10.208166+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'30","last_scrub_stamp":"2021-01-15T19:09:25.976466+0000","last_deep_scrub":"50'30","last_deep_scrub_stamp":"2021-01-15T19:09:04.507336+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.976466+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"3.16","version":"138'87","reported_seq":"371","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.204740+0000","last_change":"2021-01-15T19:10:06.361264+0000","last_active":"2021-01-15T19:10:10.204740+0000","last_peered":"2021-01-15T19:10:10.204740+0000","last_clean":"2021-01-15T19:10:10.204740+0000","last_became_active":"2021-01-15T19:09:15.343356+0000","last_became_peered":"2021-01-15T19:09:15.343356+0000","last_unstale":"2021-01-15T19:10:10.204740+0000","last_undegraded":"2021-01-15T19:10:10.204740+0000","last_fullsized":"2021-01-15T19:10:10.204740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'32","last_scrub_stamp":"2021-01-15T19:08:51.873289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.873289+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7331840,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":11,"num_read_kb":7277,"num_write":17,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2621440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"3.15","version":"140'64","reported_seq":"348","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.208169+0000","last_change":"2021-01-15T19:10:06.364293+0000","last_active":"2021-01-15T19:10:10.208169+0000","last_peered":"2021-01-15T19:10:10.208169+0000","last_clean":"2021-01-15T19:10:10.208169+0000","last_became_active":"2021-01-15T19:09:51.209994+0000","last_became_peered":"2021-01-15T19:09:51.209994+0000","last_unstale":"2021-01-15T19:10:10.208169+0000","last_undegraded":"2021-01-15T19:10:10.208169+0000","last_fullsized":"2021-01-15T19:10:10.208169+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'32","last_scrub_stamp":"2021-01-15T19:09:03.911966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:03.911966+0000","log_size":64,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5152768,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":2145,"num_write":25,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5455872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"3.14","version":"137'88","reported_seq":"286","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207839+0000","last_change":"2021-01-15T19:10:05.853218+0000","last_active":"2021-01-15T19:10:10.207839+0000","last_peered":"2021-01-15T19:10:10.207839+0000","last_clean":"2021-01-15T19:10:10.207839+0000","last_became_active":"2021-01-15T19:09:51.208908+0000","last_became_peered":"2021-01-15T19:09:51.208908+0000","last_unstale":"2021-01-15T19:10:10.207839+0000","last_undegraded":"2021-01-15T19:10:10.207839+0000","last_fullsized":"2021-01-15T19:10:10.207839+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8929280,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":12,"num_read_kb":4024,"num_write":19,"num_write_kb":11128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":21037056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":"274","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.231672+0000","last_change":"2021-01-15T19:10:06.233296+0000","last_active":"2021-01-15T19:10:10.231672+0000","last_peered":"2021-01-15T19:10:10.231672+0000","last_clean":"2021-01-15T19:10:10.231672+0000","last_became_active":"2021-01-15T19:08:43.203849+0000","last_became_peered":"2021-01-15T19:08:43.203849+0000","last_unstale":"2021-01-15T19:10:10.231672+0000","last_undegraded":"2021-01-15T19:10:10.231672+0000","last_fullsized":"2021-01-15T19:10:10.231672+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.175599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"3.12","version":"140'118","reported_seq":"479","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.555534+0000","last_change":"2021-01-15T19:10:05.852611+0000","last_active":"2021-01-15T19:10:09.555534+0000","last_peered":"2021-01-15T19:10:09.555534+0000","last_clean":"2021-01-15T19:10:09.555534+0000","last_became_active":"2021-01-15T19:09:51.213346+0000","last_became_peered":"2021-01-15T19:09:51.213346+0000","last_unstale":"2021-01-15T19:10:09.555534+0000","last_undegraded":"2021-01-15T19:10:09.555534+0000","last_fullsized":"2021-01-15T19:10:09.555534+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'42","last_scrub_stamp":"2021-01-15T19:09:02.894120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.894120+0000","log_size":118,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15769600,"num_objects":9,"num_object_clones":6,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":18,"num_read_kb":17863,"num_write":31,"num_write_kb":6456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":57647104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"1.4","version":"14'2","reported_seq":"96","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.208056+0000","last_change":"2021-01-15T19:08:53.068011+0000","last_active":"2021-01-15T19:10:10.208056+0000","last_peered":"2021-01-15T19:10:10.208056+0000","last_clean":"2021-01-15T19:10:10.208056+0000","last_became_active":"2021-01-15T19:08:52.197215+0000","last_became_peered":"2021-01-15T19:08:52.197215+0000","last_unstale":"2021-01-15T19:10:10.208056+0000","last_undegraded":"2021-01-15T19:10:10.208056+0000","last_fullsized":"2021-01-15T19:10:10.208056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"142'130","reported_seq":"472","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.236168+0000","last_change":"2021-01-15T19:10:05.848415+0000","last_active":"2021-01-15T19:10:10.236168+0000","last_peered":"2021-01-15T19:10:10.236168+0000","last_clean":"2021-01-15T19:10:10.236168+0000","last_became_active":"2021-01-15T19:08:41.227163+0000","last_became_peered":"2021-01-15T19:08:41.227163+0000","last_unstale":"2021-01-15T19:10:10.236168+0000","last_undegraded":"2021-01-15T19:10:10.236168+0000","last_fullsized":"2021-01-15T19:10:10.236168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'62","last_scrub_stamp":"2021-01-15T19:09:14.332675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:14.332675+0000","log_size":130,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11689984,"num_objects":9,"num_object_clones":7,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":9,"num_read_kb":8045,"num_write":33,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":"122","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.211823+0000","last_change":"2021-01-15T19:09:48.209380+0000","last_active":"2021-01-15T19:10:10.211823+0000","last_peered":"2021-01-15T19:10:10.211823+0000","last_clean":"2021-01-15T19:10:10.211823+0000","last_became_active":"2021-01-15T19:09:48.208922+0000","last_became_peered":"2021-01-15T19:09:48.208922+0000","last_unstale":"2021-01-15T19:10:10.211823+0000","last_undegraded":"2021-01-15T19:10:10.211823+0000","last_fullsized":"2021-01-15T19:10:10.211823+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.960545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.960545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"64'3","reported_seq":"132","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.203164+0000","last_change":"2021-01-15T19:09:51.205925+0000","last_active":"2021-01-15T19:10:09.203164+0000","last_peered":"2021-01-15T19:10:09.203164+0000","last_clean":"2021-01-15T19:10:09.203164+0000","last_became_active":"2021-01-15T19:09:51.202906+0000","last_became_peered":"2021-01-15T19:09:51.202906+0000","last_unstale":"2021-01-15T19:10:09.203164+0000","last_undegraded":"2021-01-15T19:10:09.203164+0000","last_fullsized":"2021-01-15T19:10:09.203164+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":3,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.906630+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"149","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215267+0000","last_change":"2021-01-15T19:09:54.103363+0000","last_active":"2021-01-15T19:10:09.215267+0000","last_peered":"2021-01-15T19:10:09.215267+0000","last_clean":"2021-01-15T19:10:09.215267+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:10:09.215267+0000","last_undegraded":"2021-01-15T19:10:09.215267+0000","last_fullsized":"2021-01-15T19:10:09.215267+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.103280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.103280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"137'63","reported_seq":"458","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215397+0000","last_change":"2021-01-15T19:10:06.353474+0000","last_active":"2021-01-15T19:10:09.215397+0000","last_peered":"2021-01-15T19:10:09.215397+0000","last_clean":"2021-01-15T19:10:09.215397+0000","last_became_active":"2021-01-15T19:09:51.204796+0000","last_became_peered":"2021-01-15T19:09:51.204796+0000","last_unstale":"2021-01-15T19:10:09.215397+0000","last_undegraded":"2021-01-15T19:10:09.215397+0000","last_fullsized":"2021-01-15T19:10:09.215397+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'37","last_scrub_stamp":"2021-01-15T19:08:54.947352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:54.947352+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5513216,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":36508,"num_write":6,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":8904704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"1.6","version":"14'2","reported_seq":"78","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.208114+0000","last_change":"2021-01-15T19:09:51.204380+0000","last_active":"2021-01-15T19:10:10.208114+0000","last_peered":"2021-01-15T19:10:10.208114+0000","last_clean":"2021-01-15T19:10:10.208114+0000","last_became_active":"2021-01-15T19:09:51.203386+0000","last_became_peered":"2021-01-15T19:09:51.203386+0000","last_unstale":"2021-01-15T19:10:10.208114+0000","last_undegraded":"2021-01-15T19:10:10.208114+0000","last_fullsized":"2021-01-15T19:10:10.208114+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:15.692639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:15.692639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"137'75","reported_seq":"466","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.202819+0000","last_change":"2021-01-15T19:10:05.843303+0000","last_active":"2021-01-15T19:10:09.202819+0000","last_peered":"2021-01-15T19:10:09.202819+0000","last_clean":"2021-01-15T19:10:09.202819+0000","last_became_active":"2021-01-15T19:09:51.206304+0000","last_became_peered":"2021-01-15T19:09:51.206304+0000","last_unstale":"2021-01-15T19:10:09.202819+0000","last_undegraded":"2021-01-15T19:10:09.202819+0000","last_fullsized":"2021-01-15T19:10:09.202819+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'69","last_scrub_stamp":"2021-01-15T19:10:01.942460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.942460+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5701632,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":7,"num_read_kb":1825,"num_write":4,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"143","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.231349+0000","last_change":"2021-01-15T19:08:55.065884+0000","last_active":"2021-01-15T19:10:10.231349+0000","last_peered":"2021-01-15T19:10:10.231349+0000","last_clean":"2021-01-15T19:10:10.231349+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:10:10.231349+0000","last_undegraded":"2021-01-15T19:10:10.231349+0000","last_fullsized":"2021-01-15T19:10:10.231349+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_clean_scrub_stamp":"2021-01-15T19:08:55.065810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"159","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215545+0000","last_change":"2021-01-15T19:09:28.194990+0000","last_active":"2021-01-15T19:10:09.215545+0000","last_peered":"2021-01-15T19:10:09.215545+0000","last_clean":"2021-01-15T19:10:09.215545+0000","last_became_active":"2021-01-15T19:08:52.202463+0000","last_became_peered":"2021-01-15T19:08:52.202463+0000","last_unstale":"2021-01-15T19:10:09.215545+0000","last_undegraded":"2021-01-15T19:10:09.215545+0000","last_fullsized":"2021-01-15T19:10:09.215545+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:28.194883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.194883+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"142'93","reported_seq":"443","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.212893+0000","last_change":"2021-01-15T19:10:05.335241+0000","last_active":"2021-01-15T19:10:10.212893+0000","last_peered":"2021-01-15T19:10:10.212893+0000","last_clean":"2021-01-15T19:10:10.212893+0000","last_became_active":"2021-01-15T19:09:51.207663+0000","last_became_peered":"2021-01-15T19:09:51.207663+0000","last_unstale":"2021-01-15T19:10:10.212893+0000","last_undegraded":"2021-01-15T19:10:10.212893+0000","last_fullsized":"2021-01-15T19:10:10.212893+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'47","last_scrub_stamp":"2021-01-15T19:08:53.015941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.015941+0000","log_size":93,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1359872,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":36,"num_write_kb":5144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":7340032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"168","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.231204+0000","last_change":"2021-01-15T19:09:53.454822+0000","last_active":"2021-01-15T19:10:10.231204+0000","last_peered":"2021-01-15T19:10:10.231204+0000","last_clean":"2021-01-15T19:10:10.231204+0000","last_became_active":"2021-01-15T19:09:48.201031+0000","last_became_peered":"2021-01-15T19:09:48.201031+0000","last_unstale":"2021-01-15T19:10:10.231204+0000","last_undegraded":"2021-01-15T19:10:10.231204+0000","last_fullsized":"2021-01-15T19:10:10.231204+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:53.454739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.454739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"365","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215753+0000","last_change":"2021-01-15T19:10:05.333398+0000","last_active":"2021-01-15T19:10:09.215753+0000","last_peered":"2021-01-15T19:10:09.215753+0000","last_clean":"2021-01-15T19:10:09.215753+0000","last_became_active":"2021-01-15T19:08:41.234856+0000","last_became_peered":"2021-01-15T19:08:41.234856+0000","last_unstale":"2021-01-15T19:10:09.215753+0000","last_undegraded":"2021-01-15T19:10:09.215753+0000","last_fullsized":"2021-01-15T19:10:09.215753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'18","last_scrub_stamp":"2021-01-15T19:09:16.308952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.308952+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"1.0","version":"14'2","reported_seq":"166","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.555838+0000","last_change":"2021-01-15T19:09:21.202209+0000","last_active":"2021-01-15T19:10:09.555838+0000","last_peered":"2021-01-15T19:10:09.555838+0000","last_clean":"2021-01-15T19:10:09.555838+0000","last_became_active":"2021-01-15T19:09:21.201680+0000","last_became_peered":"2021-01-15T19:09:21.201680+0000","last_unstale":"2021-01-15T19:10:09.555838+0000","last_undegraded":"2021-01-15T19:10:09.555838+0000","last_fullsized":"2021-01-15T19:10:09.555838+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:52.730150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.730150+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"143","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.231625+0000","last_change":"2021-01-15T19:09:04.092968+0000","last_active":"2021-01-15T19:10:10.231625+0000","last_peered":"2021-01-15T19:10:10.231625+0000","last_clean":"2021-01-15T19:10:10.231625+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:10:10.231625+0000","last_undegraded":"2021-01-15T19:10:10.231625+0000","last_fullsized":"2021-01-15T19:10:10.231625+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:04.092902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:04.092902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"102","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207811+0000","last_change":"2021-01-15T19:09:16.945459+0000","last_active":"2021-01-15T19:10:10.207811+0000","last_peered":"2021-01-15T19:10:10.207811+0000","last_clean":"2021-01-15T19:10:10.207811+0000","last_became_active":"2021-01-15T19:08:52.205001+0000","last_became_peered":"2021-01-15T19:08:52.205001+0000","last_unstale":"2021-01-15T19:10:10.207811+0000","last_undegraded":"2021-01-15T19:10:10.207811+0000","last_fullsized":"2021-01-15T19:10:10.207811+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:16.945387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.945387+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"133'80","reported_seq":"412","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.214943+0000","last_change":"2021-01-15T19:10:05.334048+0000","last_active":"2021-01-15T19:10:09.214943+0000","last_peered":"2021-01-15T19:10:09.214943+0000","last_clean":"2021-01-15T19:10:09.214943+0000","last_became_active":"2021-01-15T19:09:21.202949+0000","last_became_peered":"2021-01-15T19:09:21.202949+0000","last_unstale":"2021-01-15T19:10:09.214943+0000","last_undegraded":"2021-01-15T19:10:09.214943+0000","last_fullsized":"2021-01-15T19:10:09.214943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'68","last_scrub_stamp":"2021-01-15T19:09:29.177377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.177377+0000","log_size":80,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3506176,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2353,"num_write":2,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"166","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207979+0000","last_change":"2021-01-15T19:10:05.030900+0000","last_active":"2021-01-15T19:10:10.207979+0000","last_peered":"2021-01-15T19:10:10.207979+0000","last_clean":"2021-01-15T19:10:10.207979+0000","last_became_active":"2021-01-15T19:09:18.204537+0000","last_became_peered":"2021-01-15T19:09:18.204537+0000","last_unstale":"2021-01-15T19:10:10.207979+0000","last_undegraded":"2021-01-15T19:10:10.207979+0000","last_fullsized":"2021-01-15T19:10:10.207979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_clean_scrub_stamp":"2021-01-15T19:10:05.030822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"81","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215472+0000","last_change":"2021-01-15T19:09:55.119804+0000","last_active":"2021-01-15T19:10:09.215472+0000","last_peered":"2021-01-15T19:10:09.215472+0000","last_clean":"2021-01-15T19:10:09.215472+0000","last_became_active":"2021-01-15T19:09:51.205453+0000","last_became_peered":"2021-01-15T19:09:51.205453+0000","last_unstale":"2021-01-15T19:10:09.215472+0000","last_undegraded":"2021-01-15T19:10:09.215472+0000","last_fullsized":"2021-01-15T19:10:09.215472+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:55.119702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:55.119702+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"340","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207657+0000","last_change":"2021-01-15T19:10:05.347096+0000","last_active":"2021-01-15T19:10:10.207657+0000","last_peered":"2021-01-15T19:10:10.207657+0000","last_clean":"2021-01-15T19:10:10.207657+0000","last_became_active":"2021-01-15T19:08:41.228264+0000","last_became_peered":"2021-01-15T19:08:41.228264+0000","last_unstale":"2021-01-15T19:10:10.207657+0000","last_undegraded":"2021-01-15T19:10:10.207657+0000","last_fullsized":"2021-01-15T19:10:10.207657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:52.822027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.822027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"104","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.555693+0000","last_change":"2021-01-15T19:09:54.187587+0000","last_active":"2021-01-15T19:10:09.555693+0000","last_peered":"2021-01-15T19:10:09.555693+0000","last_clean":"2021-01-15T19:10:09.555693+0000","last_became_active":"2021-01-15T19:09:48.206022+0000","last_became_peered":"2021-01-15T19:09:48.206022+0000","last_unstale":"2021-01-15T19:10:09.555693+0000","last_undegraded":"2021-01-15T19:10:09.555693+0000","last_fullsized":"2021-01-15T19:10:09.555693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.187515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.187515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"185","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.556335+0000","last_change":"2021-01-15T19:09:51.213883+0000","last_active":"2021-01-15T19:10:09.556335+0000","last_peered":"2021-01-15T19:10:09.556335+0000","last_clean":"2021-01-15T19:10:09.556335+0000","last_became_active":"2021-01-15T19:09:51.213470+0000","last_became_peered":"2021-01-15T19:09:51.213470+0000","last_unstale":"2021-01-15T19:10:09.556335+0000","last_undegraded":"2021-01-15T19:10:09.556335+0000","last_fullsized":"2021-01-15T19:10:09.556335+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"137'102","reported_seq":"501","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.203199+0000","last_change":"2021-01-15T19:10:05.854081+0000","last_active":"2021-01-15T19:10:09.203199+0000","last_peered":"2021-01-15T19:10:09.203199+0000","last_clean":"2021-01-15T19:10:09.203199+0000","last_became_active":"2021-01-15T19:09:51.202675+0000","last_became_peered":"2021-01-15T19:09:51.202675+0000","last_unstale":"2021-01-15T19:10:09.203199+0000","last_undegraded":"2021-01-15T19:10:09.203199+0000","last_fullsized":"2021-01-15T19:10:09.203199+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'90","last_scrub_stamp":"2021-01-15T19:10:02.902935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.902935+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9764864,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":19,"num_read_kb":25999,"num_write":15,"num_write_kb":7208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12623872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"105","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.203071+0000","last_change":"2021-01-15T19:09:49.884000+0000","last_active":"2021-01-15T19:10:09.203071+0000","last_peered":"2021-01-15T19:10:09.203071+0000","last_clean":"2021-01-15T19:10:09.203071+0000","last_became_active":"2021-01-15T19:09:48.203471+0000","last_became_peered":"2021-01-15T19:09:48.203471+0000","last_unstale":"2021-01-15T19:10:09.203071+0000","last_undegraded":"2021-01-15T19:10:09.203071+0000","last_fullsized":"2021-01-15T19:10:09.203071+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:49.883930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:49.883930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.a","version":"14'2","reported_seq":"189","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.555951+0000","last_change":"2021-01-15T19:10:07.176493+0000","last_active":"2021-01-15T19:10:09.555951+0000","last_peered":"2021-01-15T19:10:09.555951+0000","last_clean":"2021-01-15T19:10:09.555951+0000","last_became_active":"2021-01-15T19:09:51.212693+0000","last_became_peered":"2021-01-15T19:09:51.212693+0000","last_unstale":"2021-01-15T19:10:09.555951+0000","last_undegraded":"2021-01-15T19:10:09.555951+0000","last_fullsized":"2021-01-15T19:10:09.555951+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_clean_scrub_stamp":"2021-01-15T19:10:07.176419+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"140'248","reported_seq":"575","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215207+0000","last_change":"2021-01-15T19:10:06.348293+0000","last_active":"2021-01-15T19:10:09.215207+0000","last_peered":"2021-01-15T19:10:09.215207+0000","last_clean":"2021-01-15T19:10:09.215207+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:10:09.215207+0000","last_undegraded":"2021-01-15T19:10:09.215207+0000","last_fullsized":"2021-01-15T19:10:09.215207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'109","last_scrub_stamp":"2021-01-15T19:09:15.296470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:15.296470+0000","log_size":248,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20488192,"num_objects":18,"num_object_clones":14,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":13,"num_read_kb":19615,"num_write":43,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":"106","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215275+0000","last_change":"2021-01-15T19:08:50.981704+0000","last_active":"2021-01-15T19:10:09.215275+0000","last_peered":"2021-01-15T19:10:09.215275+0000","last_clean":"2021-01-15T19:10:09.215275+0000","last_became_active":"2021-01-15T19:08:37.215228+0000","last_became_peered":"2021-01-15T19:08:37.215228+0000","last_unstale":"2021-01-15T19:10:09.215275+0000","last_undegraded":"2021-01-15T19:10:09.215275+0000","last_fullsized":"2021-01-15T19:10:09.215275+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:50.981612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:50.981612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"143'163","reported_seq":"526","reported_epoch":"143","state":"active+clean","last_fresh":"2021-01-15T19:10:11.246095+0000","last_change":"2021-01-15T19:10:06.360449+0000","last_active":"2021-01-15T19:10:11.246095+0000","last_peered":"2021-01-15T19:10:11.246095+0000","last_clean":"2021-01-15T19:10:11.246095+0000","last_became_active":"2021-01-15T19:09:15.341093+0000","last_became_peered":"2021-01-15T19:09:15.341093+0000","last_unstale":"2021-01-15T19:10:11.246095+0000","last_undegraded":"2021-01-15T19:10:11.246095+0000","last_fullsized":"2021-01-15T19:10:11.246095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'61","last_scrub_stamp":"2021-01-15T19:09:07.109680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:07.109680+0000","log_size":163,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18407424,"num_objects":13,"num_object_clones":10,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":24,"num_read_kb":30426,"num_write":75,"num_write_kb":10296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"107","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207261+0000","last_change":"2021-01-15T19:08:51.826450+0000","last_active":"2021-01-15T19:10:10.207261+0000","last_peered":"2021-01-15T19:10:10.207261+0000","last_clean":"2021-01-15T19:10:10.207261+0000","last_became_active":"2021-01-15T19:08:37.216976+0000","last_became_peered":"2021-01-15T19:08:37.216976+0000","last_unstale":"2021-01-15T19:10:10.207261+0000","last_undegraded":"2021-01-15T19:10:10.207261+0000","last_fullsized":"2021-01-15T19:10:10.207261+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:51.826360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.826360+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"14'2","reported_seq":"165","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.556199+0000","last_change":"2021-01-15T19:09:06.583849+0000","last_active":"2021-01-15T19:10:09.556199+0000","last_peered":"2021-01-15T19:10:09.556199+0000","last_clean":"2021-01-15T19:10:09.556199+0000","last_became_active":"2021-01-15T19:08:52.204479+0000","last_became_peered":"2021-01-15T19:08:52.204479+0000","last_unstale":"2021-01-15T19:10:09.556199+0000","last_undegraded":"2021-01-15T19:10:09.556199+0000","last_fullsized":"2021-01-15T19:10:09.556199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:06.583774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:06.583774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"142'158","reported_seq":"508","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.250574+0000","last_change":"2021-01-15T19:10:06.736675+0000","last_active":"2021-01-15T19:10:10.250574+0000","last_peered":"2021-01-15T19:10:10.250574+0000","last_clean":"2021-01-15T19:10:10.250574+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:10:10.250574+0000","last_undegraded":"2021-01-15T19:10:10.250574+0000","last_fullsized":"2021-01-15T19:10:10.250574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'120","last_scrub_stamp":"2021-01-15T19:09:30.324786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.324786+0000","log_size":158,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11304960,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":53,"num_read_kb":44420,"num_write":70,"num_write_kb":17216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.b","version":"0'0","reported_seq":"142","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.231785+0000","last_change":"2021-01-15T19:09:05.065723+0000","last_active":"2021-01-15T19:10:10.231785+0000","last_peered":"2021-01-15T19:10:10.231785+0000","last_clean":"2021-01-15T19:10:10.231785+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:10:10.231785+0000","last_undegraded":"2021-01-15T19:10:10.231785+0000","last_fullsized":"2021-01-15T19:10:10.231785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:05.065643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:05.065643+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"14'2","reported_seq":"169","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.555461+0000","last_change":"2021-01-15T19:09:18.403315+0000","last_active":"2021-01-15T19:10:09.555461+0000","last_peered":"2021-01-15T19:10:09.555461+0000","last_clean":"2021-01-15T19:10:09.555461+0000","last_became_active":"2021-01-15T19:08:52.203041+0000","last_became_peered":"2021-01-15T19:08:52.203041+0000","last_unstale":"2021-01-15T19:10:09.555461+0000","last_undegraded":"2021-01-15T19:10:09.555461+0000","last_fullsized":"2021-01-15T19:10:09.555461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:18.403239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.403239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"141'48","reported_seq":"440","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207830+0000","last_change":"2021-01-15T19:10:05.849571+0000","last_active":"2021-01-15T19:10:10.207830+0000","last_peered":"2021-01-15T19:10:10.207830+0000","last_clean":"2021-01-15T19:10:10.207830+0000","last_became_active":"2021-01-15T19:09:51.204375+0000","last_became_peered":"2021-01-15T19:09:51.204375+0000","last_unstale":"2021-01-15T19:10:10.207830+0000","last_undegraded":"2021-01-15T19:10:10.207830+0000","last_fullsized":"2021-01-15T19:10:10.207830+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'46","last_scrub_stamp":"2021-01-15T19:10:03.071240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:03.071240+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5029888,"num_objects":5,"num_object_clones":3,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":37,"num_read_kb":33972,"num_write":15,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":"157","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207582+0000","last_change":"2021-01-15T19:10:01.966971+0000","last_active":"2021-01-15T19:10:10.207582+0000","last_peered":"2021-01-15T19:10:10.207582+0000","last_clean":"2021-01-15T19:10:10.207582+0000","last_became_active":"2021-01-15T19:09:48.199625+0000","last_became_peered":"2021-01-15T19:09:48.199625+0000","last_unstale":"2021-01-15T19:10:10.207582+0000","last_undegraded":"2021-01-15T19:10:10.207582+0000","last_fullsized":"2021-01-15T19:10:10.207582+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:01.966888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.966888+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"142'227","reported_seq":"714","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.231902+0000","last_change":"2021-01-15T19:10:06.871906+0000","last_active":"2021-01-15T19:10:10.231902+0000","last_peered":"2021-01-15T19:10:10.231902+0000","last_clean":"2021-01-15T19:10:10.231902+0000","last_became_active":"2021-01-15T19:09:51.211452+0000","last_became_peered":"2021-01-15T19:09:51.211452+0000","last_unstale":"2021-01-15T19:10:10.231902+0000","last_undegraded":"2021-01-15T19:10:10.231902+0000","last_fullsized":"2021-01-15T19:10:10.231902+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'176","last_scrub_stamp":"2021-01-15T19:09:53.888744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.888744+0000","log_size":227,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28688384,"num_objects":17,"num_object_clones":12,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":82,"num_read_kb":111776,"num_write":104,"num_write_kb":32080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":87048192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.d","version":"0'0","reported_seq":"68","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.556228+0000","last_change":"2021-01-15T19:09:25.416216+0000","last_active":"2021-01-15T19:10:09.556228+0000","last_peered":"2021-01-15T19:10:09.556228+0000","last_clean":"2021-01-15T19:10:09.556228+0000","last_became_active":"2021-01-15T19:09:18.191906+0000","last_became_peered":"2021-01-15T19:09:18.191906+0000","last_unstale":"2021-01-15T19:10:09.556228+0000","last_undegraded":"2021-01-15T19:10:09.556228+0000","last_fullsized":"2021-01-15T19:10:09.556228+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:25.416133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.416133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"139'155","reported_seq":"503","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207981+0000","last_change":"2021-01-15T19:10:05.856895+0000","last_active":"2021-01-15T19:10:10.207981+0000","last_peered":"2021-01-15T19:10:10.207981+0000","last_clean":"2021-01-15T19:10:10.207981+0000","last_became_active":"2021-01-15T19:09:15.345068+0000","last_became_peered":"2021-01-15T19:09:15.345068+0000","last_unstale":"2021-01-15T19:10:10.207981+0000","last_undegraded":"2021-01-15T19:10:10.207981+0000","last_fullsized":"2021-01-15T19:10:10.207981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'91","last_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_deep_scrub":"101'91","last_deep_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.016550+0000","log_size":155,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8208384,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":47,"num_read_kb":12313,"num_write":85,"num_write_kb":9464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"105","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.208217+0000","last_change":"2021-01-15T19:10:02.931676+0000","last_active":"2021-01-15T19:10:10.208217+0000","last_peered":"2021-01-15T19:10:10.208217+0000","last_clean":"2021-01-15T19:10:10.208217+0000","last_became_active":"2021-01-15T19:09:48.208390+0000","last_became_peered":"2021-01-15T19:09:48.208390+0000","last_unstale":"2021-01-15T19:10:10.208217+0000","last_undegraded":"2021-01-15T19:10:10.208217+0000","last_fullsized":"2021-01-15T19:10:10.208217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:02.931607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.931607+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"141'85","reported_seq":"450","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.237891+0000","last_change":"2021-01-15T19:10:06.347699+0000","last_active":"2021-01-15T19:10:09.237891+0000","last_peered":"2021-01-15T19:10:09.237891+0000","last_clean":"2021-01-15T19:10:09.237891+0000","last_became_active":"2021-01-15T19:09:21.208779+0000","last_became_peered":"2021-01-15T19:09:21.208779+0000","last_unstale":"2021-01-15T19:10:09.237891+0000","last_undegraded":"2021-01-15T19:10:09.237891+0000","last_fullsized":"2021-01-15T19:10:09.237891+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'59","last_scrub_stamp":"2021-01-15T19:09:28.771226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.771226+0000","log_size":85,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7471104,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":14,"num_read_kb":559,"num_write":23,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.f","version":"0'0","reported_seq":"151","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.211763+0000","last_change":"2021-01-15T19:09:18.202944+0000","last_active":"2021-01-15T19:10:10.211763+0000","last_peered":"2021-01-15T19:10:10.211763+0000","last_clean":"2021-01-15T19:10:10.211763+0000","last_became_active":"2021-01-15T19:09:18.202311+0000","last_became_peered":"2021-01-15T19:09:18.202311+0000","last_unstale":"2021-01-15T19:10:10.211763+0000","last_undegraded":"2021-01-15T19:10:10.211763+0000","last_fullsized":"2021-01-15T19:10:10.211763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"137'148","reported_seq":"476","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.211948+0000","last_change":"2021-01-15T19:10:06.363701+0000","last_active":"2021-01-15T19:10:10.211948+0000","last_peered":"2021-01-15T19:10:10.211948+0000","last_clean":"2021-01-15T19:10:10.211948+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:10:10.211948+0000","last_undegraded":"2021-01-15T19:10:10.211948+0000","last_fullsized":"2021-01-15T19:10:10.211948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'93","last_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_deep_scrub":"100'93","last_deep_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.069708+0000","log_size":148,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9576448,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":30,"num_read_kb":12444,"num_write":74,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.e","version":"0'0","reported_seq":"164","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.215370+0000","last_change":"2021-01-15T19:10:04.241798+0000","last_active":"2021-01-15T19:10:09.215370+0000","last_peered":"2021-01-15T19:10:09.215370+0000","last_clean":"2021-01-15T19:10:09.215370+0000","last_became_active":"2021-01-15T19:09:18.205134+0000","last_became_peered":"2021-01-15T19:09:18.205134+0000","last_unstale":"2021-01-15T19:10:09.215370+0000","last_undegraded":"2021-01-15T19:10:09.215370+0000","last_fullsized":"2021-01-15T19:10:09.215370+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:04.241731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:04.241731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"139'94","reported_seq":"399","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207693+0000","last_change":"2021-01-15T19:10:05.849147+0000","last_active":"2021-01-15T19:10:10.207693+0000","last_peered":"2021-01-15T19:10:10.207693+0000","last_clean":"2021-01-15T19:10:10.207693+0000","last_became_active":"2021-01-15T19:09:51.206324+0000","last_became_peered":"2021-01-15T19:09:51.206324+0000","last_unstale":"2021-01-15T19:10:10.207693+0000","last_undegraded":"2021-01-15T19:10:10.207693+0000","last_fullsized":"2021-01-15T19:10:10.207693+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":94,"ondisk_log_size":94,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9035776,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":17,"num_read_kb":26808,"num_write":40,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]},{"pgid":"2.10","version":"0'0","reported_seq":"110","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.214994+0000","last_change":"2021-01-15T19:10:02.233511+0000","last_active":"2021-01-15T19:10:09.214994+0000","last_peered":"2021-01-15T19:10:09.214994+0000","last_clean":"2021-01-15T19:10:09.214994+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:10:09.214994+0000","last_undegraded":"2021-01-15T19:10:09.214994+0000","last_fullsized":"2021-01-15T19:10:09.214994+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:02.233439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.233439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"115","reported_epoch":"142","state":"active+clean","last_fresh":"2021-01-15T19:10:10.207343+0000","last_change":"2021-01-15T19:09:12.353642+0000","last_active":"2021-01-15T19:10:10.207343+0000","last_peered":"2021-01-15T19:10:10.207343+0000","last_clean":"2021-01-15T19:10:10.207343+0000","last_became_active":"2021-01-15T19:09:12.353214+0000","last_became_peered":"2021-01-15T19:09:12.353214+0000","last_unstale":"2021-01-15T19:10:10.207343+0000","last_undegraded":"2021-01-15T19:10:10.207343+0000","last_fullsized":"2021-01-15T19:10:10.207343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.902685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.902685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"139'145","reported_seq":"480","reported_epoch":"141","state":"active+clean","last_fresh":"2021-01-15T19:10:09.555630+0000","last_change":"2021-01-15T19:10:05.340803+0000","last_active":"2021-01-15T19:10:09.555630+0000","last_peered":"2021-01-15T19:10:09.555630+0000","last_clean":"2021-01-15T19:10:09.555630+0000","last_became_active":"2021-01-15T19:09:21.202752+0000","last_became_peered":"2021-01-15T19:09:21.202752+0000","last_unstale":"2021-01-15T19:10:09.555630+0000","last_undegraded":"2021-01-15T19:10:09.555630+0000","last_fullsized":"2021-01-15T19:10:09.555630+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'67","last_scrub_stamp":"2021-01-15T19:08:49.729161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:49.729161+0000","log_size":145,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5316608,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":27,"num_read_kb":26360,"num_write":11,"num_write_kb":3856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":12124160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":206094336,"num_objects":167,"num_object_clones":121,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":10,"num_read":457,"num_read_kb":426971,"num_write":743,"num_write_kb":173440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":248659968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":378990592,"data_stored":643452928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3805,"internal_metadata":0},"log_size":2542,"ondisk_log_size":2542,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9184,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":22,"acting":22,"num_store_stats":8},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116532,"kb_used_data":82668,"kb_used_omap":0,"kb_used_meta":33856,"kb_avail":94255308,"statfs":{"total":96636764160,"available":96517435392,"internally_reserved":0,"allocated":84652032,"data_stored":157845246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34668544},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91400000000000003,"15min":0.92000000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1519999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.26}]},{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97499999999999998,"15min":0.97099999999999997},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.341,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.101},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91400000000000003,"15min":0.91000000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.0009999999999999}]},{"osd":2,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.98499999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.339},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91200000000000003,"15min":0.90900000000000003},"min":{"1min":0.73999999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1140000000000001,"5min":1.276,"15min":1.276},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0800000000000001,"15min":1.0900000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1639999999999999,"15min":1.165},"min":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.127}]},{"osd":4,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.97099999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.204,"5min":1.427,"15min":1.427},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.206,"15min":1.23},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.667,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.97399999999999998}]},{"osd":5,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.2150000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.466,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.131,"15min":1.1479999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.387,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.075}]}]},{"osd":6,"up_from":0,"seq":566935683075,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77268,"kb_used_data":49420,"kb_used_omap":0,"kb_used_meta":27840,"kb_avail":94294572,"statfs":{"total":96636764160,"available":96557641728,"internally_reserved":0,"allocated":50606080,"data_stored":82967743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":28508160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2729999999999999,"5min":1.452,"15min":1.452},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0169999999999999,"15min":1.0469999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1819999999999999,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":0.92400000000000004}]},{"osd":2,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.83599999999999997,"15min":0.85499999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.008,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.085,"15min":1.0920000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3480000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.1850000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:09:55 2021","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1479999999999999,"15min":1.1719999999999999},"min":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4910000000000001,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.028},{"interface":"front","average":{"1min":1.111,"5min":1.427,"15min":1.4790000000000001},"min":{"1min":0.70199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6519999999999999,"5min":10.584,"15min":10.584},"last":1.254}]},{"osd":4,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.3620000000000001,"15min":1.3879999999999999},"min":{"1min":0.755,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1699999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.105}]},{"osd":5,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.5469999999999999,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.085},{"interface":"front","average":{"1min":1.228,"5min":1.391,"15min":1.4179999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.673,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.375}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.3260000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.488,"15min":1.512},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.7629999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.587,"15min":1.621},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0150000000000001,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.5129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70176,"kb_used_data":45336,"kb_used_omap":0,"kb_used_meta":24832,"kb_avail":94301664,"statfs":{"total":96636764160,"available":96564903936,"internally_reserved":0,"allocated":46424064,"data_stored":73193214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":25427968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.048,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.91000000000000003}]},{"osd":2,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94199999999999995,"15min":0.95299999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.103},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.071}]},{"osd":3,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0549999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.181,"15min":1.198},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.506,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.405}]},{"osd":4,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5369999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.1699999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.681,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.3580000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.724,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.4370000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4350000000000001,"15min":1.4550000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.6819999999999999,"5min":1.96,"15min":1.96},"last":1.522},{"interface":"front","average":{"1min":1.367,"5min":1.5349999999999999,"15min":1.5629999999999999},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":1.847,"5min":2.024,"15min":2.024},"last":1.385}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98340,"kb_used_data":65052,"kb_used_omap":0,"kb_used_meta":33280,"kb_avail":94273500,"statfs":{"total":96636764160,"available":96536064000,"internally_reserved":0,"allocated":66613248,"data_stored":116419775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34078720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.54700000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2629999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.63}]},{"osd":1,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95699999999999996,"15min":0.94899999999999995},"min":{"1min":0.76900000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.82099999999999995,"15min":0.81100000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.089,"15min":1.0900000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.339}]},{"osd":4,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.149,"15min":1.141},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.86299999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.538,"5min":1.859,"15min":1.859},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.29,"15min":1.2829999999999999},"min":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9239999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.222},{"interface":"front","average":{"1min":1.165,"5min":1.204,"15min":1.21},"min":{"1min":0.90100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.468,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.365}]},{"osd":7,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3620000000000001,"15min":1.355},"min":{"1min":0.92300000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0510000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.492},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3999999999999999,"15min":1.397},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.2789999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26528,"kb_used_data":9496,"kb_used_omap":0,"kb_used_meta":17024,"kb_avail":94345312,"statfs":{"total":96636764160,"available":96609599488,"internally_reserved":0,"allocated":9723904,"data_stored":16541438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":17432576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.87,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0629999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.052,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.1359999999999999}]},{"osd":2,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.421,"5min":1.843,"15min":1.843},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.86199999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4330000000000001,"5min":1.911,"15min":1.911},"last":1.278}]},{"osd":4,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.81399999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.429,"5min":2.02,"15min":2.02},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1659999999999999,"15min":1.1779999999999999},"min":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4870000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3049999999999999,"15min":1.3089999999999999},"min":{"1min":0.98699999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.097},{"interface":"front","average":{"1min":1.29,"5min":1.4490000000000001,"15min":1.4750000000000001},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.7410000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.534}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.389,"15min":1.405},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6299999999999999,"5min":2.097,"15min":2.097},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.4139999999999999,"15min":1.4219999999999999},"min":{"1min":1.095,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.395}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91116,"kb_used_data":57124,"kb_used_omap":0,"kb_used_meta":33984,"kb_avail":94280724,"statfs":{"total":96636764160,"available":96543461376,"internally_reserved":0,"allocated":58494976,"data_stored":112113406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":34799616},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3280000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.252,"5min":1.409,"15min":1.409},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91400000000000003,"15min":0.90600000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.617,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2050000000000001,"15min":1.1990000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.014}]},{"osd":2,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.98299999999999998,"15min":0.99399999999999999},"min":{"1min":0.69699999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2050000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.383},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5980000000000001,"5min":2.161,"15min":2.161},"last":1.0780000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1950000000000001,"15min":1.1990000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.137,"15min":1.139},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.879}]},{"osd":5,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3169999999999999,"5min":1.571,"15min":1.571},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.2989999999999999,"15min":1.331},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.90500000000000003}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.514,"15min":1.556},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.2440000000000002,"5min":2.52,"15min":2.52},"last":1.147},{"interface":"front","average":{"1min":1.202,"5min":1.365,"15min":1.3919999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.80000000000000004}]}]},{"osd":5,"up_from":0,"seq":55834574879,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}]}} 2021-01-15T19:10:14.262 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set rbd pgp_num 18 2021-01-15T19:10:14.300 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:14.299+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:14.373 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:14.373 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:14.383 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:14.401 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:14.400+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:14.502 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:14.501+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:14.549 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:14.549 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:14.558 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:14.603 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:14.602+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:14.704 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:14.703+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:14.722 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:14.722 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:14.735 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:14.741 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:14.739+0000 7fb6b1e51700 -1 osd.0 pg_epoch: 146 pg[3.14s0( v 146'90 (0'0,146'90] local-lis/les=122/123 n=5 ec=52/21 lis/c=122/122 les/c/f=123/123/0 sis=122) [0,2,1]p0(0) r=0 lpr=122 luod=137'88 crt=137'88 lcod 135'86 mlcod 135'86 active+clean+snaptrim trimq=[26~1] ps=[19~1,2a~1,2f~1,38~1,3a~1,3e~1,41~1,46~1,4c~1,50~1,53~2,57~3,5c~2,62~1,6d~1]] removing snap head 2021-01-15T19:10:14.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:14.804 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:14.803+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:14.862 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:14.863 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:14.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:14.900 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:14.905 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:14.904+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:14.920 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:14.940 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:14.960 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:14.981 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:15.002 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:15.006 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:15.005+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:15.025 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:15.041 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:15.042 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:15.046 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:15.047 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.047 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.048 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.048 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.048 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.048 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.048 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.049 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:15.052 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:15.136 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:15.135+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:15.208 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:15.207+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:15.219 INFO:tasks.rados.rados.0.smithi001.stdout:587: done (1 left) 2021-01-15T19:10:15.219 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 6 current snap is 66 2021-01-15T19:10:15.220 INFO:tasks.rados.rados.0.smithi001.stdout:588: seq_num 209 ranges {0=704512,704512=401408} 2021-01-15T19:10:15.220 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:15.220 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:15.221 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:15.221 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:15.221 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:15.222 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 2 pgp_num to 18 2021-01-15T19:10:15.223 INFO:tasks.rados.rados.0.smithi001.stdout:588: writing smithi00140323-6 from 0 to 704512 tid 1 2021-01-15T19:10:15.226 INFO:tasks.rados.rados.0.smithi001.stdout:588: writing smithi00140323-6 from 704512 to 1105920 tid 2 2021-01-15T19:10:15.232 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:15.234 INFO:tasks.rados.rados.0.smithi001.stdout:584: expect (ObjNum 103 snap 17 seq_num 103) 2021-01-15T19:10:15.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:10:15.250 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:15.247+0000 7fed3ef45700 -1 osd.1 pg_epoch: 147 pg[3.10s0( v 147'156 (0'0,147'156] local-lis/les=92/93 n=7 ec=52/21 lis/c=92/92 les/c/f=93/93/0 sis=92) [1,2,4]p1(0) r=0 lpr=92 luod=147'153 crt=147'153 lcod 147'152 mlcod 147'152 active+clean+snaptrim trimq=[44~1] ps=[19~1,26~1,2a~1,2f~1,38~1,3a~1,3e~1,41~1,46~1,4c~1,50~1,53~2,57~3,5c~2,62~1,6d~1]] removing snap head 2021-01-15T19:10:15.316 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:15.315+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:15.397 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:15.397 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:15.398 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:15.398 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:15.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:15.431 INFO:tasks.rados.rados.0.smithi001.stdout:588: finishing write tid 1 to smithi00140323-6 2021-01-15T19:10:15.431 INFO:tasks.rados.rados.0.smithi001.stdout:588: finishing write tid 2 to smithi00140323-6 2021-01-15T19:10:15.432 INFO:tasks.rados.rados.0.smithi001.stdout:588: finishing write tid 3 to smithi00140323-6 2021-01-15T19:10:15.432 INFO:tasks.rados.rados.0.smithi001.stdout:588: finishing write tid 4 to smithi00140323-6 2021-01-15T19:10:15.432 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 6 v 154 (ObjNum 209 snap 66 seq_num 209) dirty exists 2021-01-15T19:10:15.432 INFO:tasks.rados.rados.0.smithi001.stdout:588: left oid 6 (ObjNum 209 snap 66 seq_num 209) 2021-01-15T19:10:15.433 INFO:tasks.rados.rados.0.smithi001.stdout:584: done (1 left) 2021-01-15T19:10:15.433 INFO:tasks.rados.rados.0.smithi001.stdout:588: done (0 left) 2021-01-15T19:10:15.433 INFO:tasks.rados.rados.0.smithi001.stdout:589: read oid 47 snap -1 2021-01-15T19:10:15.433 INFO:tasks.rados.rados.0.smithi001.stdout:589: expect deleted 2021-01-15T19:10:15.434 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 14 current snap is 66 2021-01-15T19:10:15.434 INFO:tasks.rados.rados.0.smithi001.stdout:590: seq_num 210 ranges {1359872=720896,2080768=401408} 2021-01-15T19:10:15.442 INFO:tasks.rados.rados.0.smithi001.stdout:590: writing smithi00140323-14 from 1359872 to 2080768 tid 1 2021-01-15T19:10:15.444 INFO:tasks.rados.rados.0.smithi001.stdout:590: writing smithi00140323-14 from 2080768 to 2482176 tid 2 2021-01-15T19:10:15.445 INFO:tasks.rados.rados.0.smithi001.stdout:591: delete oid 41 current snap is 66 2021-01-15T19:10:15.448 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:15.447+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:15.449 INFO:tasks.rados.rados.0.smithi001.stdout:589: done (2 left) 2021-01-15T19:10:15.450 INFO:tasks.rados.rados.0.smithi001.stdout:591: done (1 left) 2021-01-15T19:10:15.450 INFO:tasks.rados.rados.0.smithi001.stdout:592: read oid 40 snap -1 2021-01-15T19:10:15.450 INFO:tasks.rados.rados.0.smithi001.stdout:592: expect deleted 2021-01-15T19:10:15.451 INFO:tasks.rados.rados.0.smithi001.stdout:593: snap_remove snap 63 2021-01-15T19:10:15.513 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:15.512+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:15.581 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:15.582 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:15.582 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:15.583 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:15.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:15.626 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:15.623+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:15.719 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:15.717+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:15.769 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:15.771 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:15.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:15.824 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:15.822+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:15.919 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:15.917+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:15.959 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.702377944000006, 2021-01-15T19:10:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.702212830999997, 2021-01-15T19:10:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.694932781999995, 2021-01-15T19:10:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.694806149000001, 2021-01-15T19:10:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.612069184000006, 2021-01-15T19:10:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.611919569999998, 2021-01-15T19:10:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.591621887999999, 2021-01-15T19:10:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.588565236999997, 2021-01-15T19:10:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.585551975000001, 2021-01-15T19:10:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.582804363999998, 2021-01-15T19:10:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.582370542, 2021-01-15T19:10:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.582246382999998, 2021-01-15T19:10:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.477569848000002, 2021-01-15T19:10:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.597348769, 2021-01-15T19:10:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.567630612000002, 2021-01-15T19:10:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.563544444999998, 2021-01-15T19:10:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 62.563408694000003, 2021-01-15T19:10:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.724909710999999, 2021-01-15T19:10:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.750330247, 2021-01-15T19:10:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 5.7501985859999998, 2021-01-15T19:10:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.302 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:16.304 INFO:tasks.rados.rados.0.smithi001.stdout:593: done (2 left) 2021-01-15T19:10:16.304 INFO:tasks.rados.rados.0.smithi001.stdout:594: snap_create 2021-01-15T19:10:16.304 INFO:tasks.rados.rados.0.smithi001.stdout:590: finishing write tid 1 to smithi00140323-14 2021-01-15T19:10:16.305 INFO:tasks.rados.rados.0.smithi001.stdout:590: finishing write tid 2 to smithi00140323-14 2021-01-15T19:10:16.305 INFO:tasks.rados.rados.0.smithi001.stdout:590: finishing write tid 3 to smithi00140323-14 2021-01-15T19:10:16.305 INFO:tasks.rados.rados.0.smithi001.stdout:590: finishing write tid 4 to smithi00140323-14 2021-01-15T19:10:16.305 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 99 (ObjNum 210 snap 66 seq_num 210) dirty exists 2021-01-15T19:10:16.306 INFO:tasks.rados.rados.0.smithi001.stdout:590: left oid 14 (ObjNum 210 snap 66 seq_num 210) 2021-01-15T19:10:16.306 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:10:16.306 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:16.302+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:16.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:10:16.405 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:16.404+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:16.478 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:16.479 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:16.479 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:16.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:10:16.506 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:16.504+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:16.607 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:16.605+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:16.620 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:16.621 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:16.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:10:16.707 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:16.706+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:16.767 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:10:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 146.24773847500001, 2021-01-15T19:10:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:10:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:10:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:10:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:10:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:10:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:10:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:10:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:10:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:10:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:10:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:10:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:10:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:10:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:10:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:10:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:10:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:10:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:10:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:10:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:10:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:10:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:829 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.582024154999999, 2021-01-15T19:10:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001, 2021-01-15T19:10:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 829 2021-01-15T19:10:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4360000000000001e-05 2021-01-15T19:10:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0169999999999999e-06 2021-01-15T19:10:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182630+0000", 2021-01-15T19:10:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4198e-05 2021-01-15T19:10:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182644+0000", 2021-01-15T19:10:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1253999999999999e-05 2021-01-15T19:10:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182656+0000", 2021-01-15T19:10:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9756999999999999e-05 2021-01-15T19:10:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182695+0000", 2021-01-15T19:10:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2819999999999996e-06 2021-01-15T19:10:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182704+0000", 2021-01-15T19:10:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048051730000000003 2021-01-15T19:10:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187509+0000", 2021-01-15T19:10:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114002 2021-01-15T19:10:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187623+0000", 2021-01-15T19:10:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012123558 2021-01-15T19:10:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.199746+0000", 2021-01-15T19:10:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001 2021-01-15T19:10:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=315, reqid=client.4418.0:833, at_version=80'84, trim_to=0'0, roll_forward_to=78'82))", 2021-01-15T19:10:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.577462187999998, 2021-01-15T19:10:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001, 2021-01-15T19:10:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6477000000000002e-05 2021-01-15T19:10:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187187+0000", 2021-01-15T19:10:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4749e-05 2021-01-15T19:10:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187212+0000", 2021-01-15T19:10:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5977e-05 2021-01-15T19:10:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187228+0000", 2021-01-15T19:10:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0770000000000008e-06 2021-01-15T19:10:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187236+0000", 2021-01-15T19:10:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014010800000000001 2021-01-15T19:10:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187376+0000", 2021-01-15T19:10:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8624999999999999e-05 2021-01-15T19:10:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187395+0000", 2021-01-15T19:10:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013611689 2021-01-15T19:10:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201006+0000", 2021-01-15T19:10:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7980999999999999e-05 2021-01-15T19:10:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201064+0000", 2021-01-15T19:10:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001 2021-01-15T19:10:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:971 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.555318794000002, 2021-01-15T19:10:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002, 2021-01-15T19:10:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 971 2021-01-15T19:10:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2442999999999998e-05 2021-01-15T19:10:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209337+0000", 2021-01-15T19:10:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0950000000000002e-06 2021-01-15T19:10:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209346+0000", 2021-01-15T19:10:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9199999999999999e-05 2021-01-15T19:10:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209365+0000", 2021-01-15T19:10:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5750000000000007e-06 2021-01-15T19:10:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209374+0000", 2021-01-15T19:10:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074262999999999998 2021-01-15T19:10:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210117+0000", 2021-01-15T19:10:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022372900000000001 2021-01-15T19:10:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210341+0000", 2021-01-15T19:10:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016740338 2021-01-15T19:10:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227081+0000", 2021-01-15T19:10:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002 2021-01-15T19:10:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:972 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.555172038000002, 2021-01-15T19:10:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999, 2021-01-15T19:10:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 972 2021-01-15T19:10:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5875000000000003e-05 2021-01-15T19:10:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209477+0000", 2021-01-15T19:10:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7499999999999992e-06 2021-01-15T19:10:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209486+0000", 2021-01-15T19:10:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4825e-05 2021-01-15T19:10:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209500+0000", 2021-01-15T19:10:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209508+0000", 2021-01-15T19:10:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012352109999999999 2021-01-15T19:10:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210743+0000", 2021-01-15T19:10:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012467700000000001 2021-01-15T19:10:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210867+0000", 2021-01-15T19:10:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018560825 2021-01-15T19:10:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.229428+0000", 2021-01-15T19:10:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999 2021-01-15T19:10:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:973 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.555040706, 2021-01-15T19:10:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001, 2021-01-15T19:10:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 973 2021-01-15T19:10:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7021999999999997e-05 2021-01-15T19:10:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209609+0000", 2021-01-15T19:10:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3140000000000004e-06 2021-01-15T19:10:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209618+0000", 2021-01-15T19:10:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4259e-05 2021-01-15T19:10:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209632+0000", 2021-01-15T19:10:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2021-01-15T19:10:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209639+0000", 2021-01-15T19:10:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039100000000000003 2021-01-15T19:10:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213549+0000", 2021-01-15T19:10:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124948 2021-01-15T19:10:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213674+0000", 2021-01-15T19:10:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018190644999999998 2021-01-15T19:10:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.231864+0000", 2021-01-15T19:10:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001 2021-01-15T19:10:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1093 3.10s0 3:095f0b32:::smithi00140323-47:head [rollback 50] snapc 50=[50,4b,44,41,26] ondisk+write+known_if_redirected e109)", 2021-01-15T19:10:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.528366573, 2021-01-15T19:10:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242, 2021-01-15T19:10:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1093 2021-01-15T19:10:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3699999999999998e-05 2021-01-15T19:10:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236270+0000", 2021-01-15T19:10:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4700000000000008e-06 2021-01-15T19:10:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236280+0000", 2021-01-15T19:10:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6716999999999999e-05 2021-01-15T19:10:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236296+0000", 2021-01-15T19:10:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5490000000000006e-06 2021-01-15T19:10:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236305+0000", 2021-01-15T19:10:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4991999999999998e-05 2021-01-15T19:10:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236360+0000", 2021-01-15T19:10:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3660000000000003e-06 2021-01-15T19:10:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236367+0000", 2021-01-15T19:10:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005477549 2021-01-15T19:10:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241845+0000", 2021-01-15T19:10:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137458 2021-01-15T19:10:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241982+0000", 2021-01-15T19:10:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097224410000000001 2021-01-15T19:10:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.251705+0000", 2021-01-15T19:10:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242 2021-01-15T19:10:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 109/95 ECSubWrite(tid=263, reqid=client.4418.0:1090, at_version=109'60, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.524550935000001, 2021-01-15T19:10:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999, 2021-01-15T19:10:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8936e-05 2021-01-15T19:10:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240091+0000", 2021-01-15T19:10:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1778000000000001e-05 2021-01-15T19:10:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240113+0000", 2021-01-15T19:10:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6714999999999998e-05 2021-01-15T19:10:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240129+0000", 2021-01-15T19:10:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240137+0000", 2021-01-15T19:10:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9104999999999998e-05 2021-01-15T19:10:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240186+0000", 2021-01-15T19:10:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240194+0000", 2021-01-15T19:10:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017479262999999998 2021-01-15T19:10:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257674+0000", 2021-01-15T19:10:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9951999999999999e-05 2021-01-15T19:10:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257724+0000", 2021-01-15T19:10:16.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999 2021-01-15T19:10:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 110/95 ECSubWrite(tid=269, reqid=client.4418.0:1098, at_version=110'64, trim_to=0'0, roll_forward_to=109'62))", 2021-01-15T19:10:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.570489924999997, 2021-01-15T19:10:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999, 2021-01-15T19:10:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5725e-05 2021-01-15T19:10:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194149+0000", 2021-01-15T19:10:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3603000000000003e-05 2021-01-15T19:10:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194182+0000", 2021-01-15T19:10:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0721e-05 2021-01-15T19:10:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194203+0000", 2021-01-15T19:10:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3929999999999997e-06 2021-01-15T19:10:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194212+0000", 2021-01-15T19:10:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0593000000000006e-05 2021-01-15T19:10:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194293+0000", 2021-01-15T19:10:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3596e-05 2021-01-15T19:10:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194307+0000", 2021-01-15T19:10:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017420983000000001 2021-01-15T19:10:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211728+0000", 2021-01-15T19:10:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6654999999999999e-05 2021-01-15T19:10:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211784+0000", 2021-01-15T19:10:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999 2021-01-15T19:10:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1116 3.4s1 3:21d37027:::smithi00140323-27:head [copy-from ver 54] snapc 57=[57,56,53,50,4b,44,41,26] ondisk+write+known_if_redirected e114)", 2021-01-15T19:10:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.512051323000001, 2021-01-15T19:10:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999, 2021-01-15T19:10:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1116 2021-01-15T19:10:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3304000000000002e-05 2021-01-15T19:10:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252605+0000", 2021-01-15T19:10:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0723000000000001e-05 2021-01-15T19:10:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252616+0000", 2021-01-15T19:10:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0720000000000002e-05 2021-01-15T19:10:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252636+0000", 2021-01-15T19:10:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1799e-05 2021-01-15T19:10:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252648+0000", 2021-01-15T19:10:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5387000000000003e-05 2021-01-15T19:10:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252714+0000", 2021-01-15T19:10:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029657199999999999 2021-01-15T19:10:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253010+0000", 2021-01-15T19:10:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030464359 2021-01-15T19:10:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.283475+0000", 2021-01-15T19:10:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011930975999999999 2021-01-15T19:10:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295405+0000", 2021-01-15T19:10:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2021-01-15T19:10:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295488+0000", 2021-01-15T19:10:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074145999999999999 2021-01-15T19:10:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296229+0000", 2021-01-15T19:10:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999 2021-01-15T19:10:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1117 3.4s1 3:21d37027:::smithi00140323-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e114)", 2021-01-15T19:10:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.511886826999998, 2021-01-15T19:10:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003, 2021-01-15T19:10:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1117 2021-01-15T19:10:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3238000000000002e-05 2021-01-15T19:10:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252749+0000", 2021-01-15T19:10:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8820000000000001e-06 2021-01-15T19:10:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252757+0000", 2021-01-15T19:10:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6419e-05 2021-01-15T19:10:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252774+0000", 2021-01-15T19:10:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9879999999999998e-06 2021-01-15T19:10:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252781+0000", 2021-01-15T19:10:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051378399999999999 2021-01-15T19:10:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253294+0000", 2021-01-15T19:10:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8572000000000006e-05 2021-01-15T19:10:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253373+0000", 2021-01-15T19:10:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040965935000000002 2021-01-15T19:10:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294339+0000", 2021-01-15T19:10:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8033000000000001e-05 2021-01-15T19:10:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294437+0000", 2021-01-15T19:10:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018291189999999999 2021-01-15T19:10:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296266+0000", 2021-01-15T19:10:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0880999999999999e-05 2021-01-15T19:10:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296357+0000", 2021-01-15T19:10:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016417 2021-01-15T19:10:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296521+0000", 2021-01-15T19:10:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003 2021-01-15T19:10:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s0 3.18aae252 (undecoded) ondisk+write+peerstat_old+known_if_redirected e119)", 2021-01-15T19:10:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.485250493999999, 2021-01-15T19:10:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263, 2021-01-15T19:10:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:10:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5269000000000003e-05 2021-01-15T19:10:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279388+0000", 2021-01-15T19:10:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4849999999999998e-06 2021-01-15T19:10:16.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279397+0000", 2021-01-15T19:10:16.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.63e-05 2021-01-15T19:10:16.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279414+0000", 2021-01-15T19:10:16.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:16.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279422+0000", 2021-01-15T19:10:16.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5129e-05 2021-01-15T19:10:16.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279477+0000", 2021-01-15T19:10:16.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1559999999999997e-06 2021-01-15T19:10:16.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279481+0000", 2021-01-15T19:10:16.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.92165986799999999 2021-01-15T19:10:16.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201141+0000", 2021-01-15T19:10:16.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9066999999999999e-05 2021-01-15T19:10:16.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201160+0000", 2021-01-15T19:10:16.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263 2021-01-15T19:10:16.973 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.973 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4166.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e99)", 2021-01-15T19:10:16.975 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:16.975 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 29.208881270999999, 2021-01-15T19:10:16.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116, 2021-01-15T19:10:16.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:16.976 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.976 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4166", 2021-01-15T19:10:16.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4191424063", 2021-01-15T19:10:16.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4 2021-01-15T19:10:16.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.978 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:16.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:16.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7705999999999999e-05 2021-01-15T19:10:16.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555739+0000", 2021-01-15T19:10:16.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8659999999999995e-06 2021-01-15T19:10:16.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:16.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555748+0000", 2021-01-15T19:10:16.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2021-01-15T19:10:16.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:16.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555761+0000", 2021-01-15T19:10:16.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9859999999999994e-06 2021-01-15T19:10:16.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:16.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555769+0000", 2021-01-15T19:10:16.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034585600000000002 2021-01-15T19:10:16.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556115+0000", 2021-01-15T19:10:16.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:16.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:16.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556129+0000", 2021-01-15T19:10:16.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.65062051300000001 2021-01-15T19:10:16.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:16.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206749+0000", 2021-01-15T19:10:16.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014163399999999999 2021-01-15T19:10:16.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:16.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206891+0000", 2021-01-15T19:10:16.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116 2021-01-15T19:10:16.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.993 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:16.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:16.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1187 3.12s0 3:4bacf681:::smithi00140323-17:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:16.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:16.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.538207925999998, 2021-01-15T19:10:16.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999, 2021-01-15T19:10:16.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:16.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:16.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:16.995 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:16.996 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:16.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1187 2021-01-15T19:10:16.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.996 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:16.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:16.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:16.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:16.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:16.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:16.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8656999999999998e-05 2021-01-15T19:10:16.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:16.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:16.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:16.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226444+0000", 2021-01-15T19:10:17.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:17.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226452+0000", 2021-01-15T19:10:17.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3689000000000001e-05 2021-01-15T19:10:17.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226466+0000", 2021-01-15T19:10:17.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8540000000000002e-06 2021-01-15T19:10:17.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226472+0000", 2021-01-15T19:10:17.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2799999999999997e-05 2021-01-15T19:10:17.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226514+0000", 2021-01-15T19:10:17.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1725999999999999e-05 2021-01-15T19:10:17.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:17.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226526+0000", 2021-01-15T19:10:17.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066831169999999997 2021-01-15T19:10:17.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.233209+0000", 2021-01-15T19:10:17.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011647090000000001 2021-01-15T19:10:17.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234374+0000", 2021-01-15T19:10:17.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00061833799999999996 2021-01-15T19:10:17.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:17.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234992+0000", 2021-01-15T19:10:17.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001433157 2021-01-15T19:10:17.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:17.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.236425+0000", 2021-01-15T19:10:17.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044118559999999996 2021-01-15T19:10:17.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:17.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240837+0000", 2021-01-15T19:10:17.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152409 2021-01-15T19:10:17.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240990+0000", 2021-01-15T19:10:17.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999 2021-01-15T19:10:17.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.013 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.014 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.014 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1188 3.12s0 3:4bacf681:::smithi00140323-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:17.014 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:17.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.538072666000001, 2021-01-15T19:10:17.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001, 2021-01-15T19:10:17.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:17.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:17.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:17.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:17.016 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:17.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1188 2021-01-15T19:10:17.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.017 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:17.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:17.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:17.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:17.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:17.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:17.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0703e-05 2021-01-15T19:10:17.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:17.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226561+0000", 2021-01-15T19:10:17.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5660000000000001e-06 2021-01-15T19:10:17.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226569+0000", 2021-01-15T19:10:17.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3385e-05 2021-01-15T19:10:17.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226582+0000", 2021-01-15T19:10:17.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:17.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226587+0000", 2021-01-15T19:10:17.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.409e-05 2021-01-15T19:10:17.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226632+0000", 2021-01-15T19:10:17.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1242e-05 2021-01-15T19:10:17.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:17.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226643+0000", 2021-01-15T19:10:17.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086436099999999995 2021-01-15T19:10:17.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235286+0000", 2021-01-15T19:10:17.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016845599999999999 2021-01-15T19:10:17.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:17.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235455+0000", 2021-01-15T19:10:17.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060319140000000002 2021-01-15T19:10:17.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241487+0000", 2021-01-15T19:10:17.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014580699999999999 2021-01-15T19:10:17.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241633+0000", 2021-01-15T19:10:17.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001527306 2021-01-15T19:10:17.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.243160+0000", 2021-01-15T19:10:17.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001 2021-01-15T19:10:17.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.036 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.036 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.037 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1194 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:17.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:17.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 19.508329707000001, 2021-01-15T19:10:17.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019, 2021-01-15T19:10:17.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:17.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:17.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:17.040 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:17.040 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:17.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1194 2021-01-15T19:10:17.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.041 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:17.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:17.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:17.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:17.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:17.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:17.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8844999999999998e-05 2021-01-15T19:10:17.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:17.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256302+0000", 2021-01-15T19:10:17.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9030000000000004e-06 2021-01-15T19:10:17.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256310+0000", 2021-01-15T19:10:17.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4749999999999999e-05 2021-01-15T19:10:17.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256325+0000", 2021-01-15T19:10:17.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2370000000000003e-06 2021-01-15T19:10:17.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256332+0000", 2021-01-15T19:10:17.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011409671999999999 2021-01-15T19:10:17.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267742+0000", 2021-01-15T19:10:17.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017902200000000001 2021-01-15T19:10:17.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:17.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267921+0000", 2021-01-15T19:10:17.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027501040000000002 2021-01-15T19:10:17.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270671+0000", 2021-01-15T19:10:17.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020387700000000001 2021-01-15T19:10:17.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270875+0000", 2021-01-15T19:10:17.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012356089999999999 2021-01-15T19:10:17.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.272110+0000", 2021-01-15T19:10:17.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019 2021-01-15T19:10:17.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.057 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1404 3.10s0 3:0e97918f:::smithi00140323-6:head [append 0~704512] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:17.058 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:17.058 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.5401847689999999, 2021-01-15T19:10:17.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02076683, 2021-01-15T19:10:17.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:17.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:17.060 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:17.060 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:17.060 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:17.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1404 2021-01-15T19:10:17.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.062 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:17.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:17.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:17.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:17.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:17.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:17.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033569900000000001 2021-01-15T19:10:17.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:17.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224744+0000", 2021-01-15T19:10:17.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:10:17.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224753+0000", 2021-01-15T19:10:17.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6562000000000002e-05 2021-01-15T19:10:17.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224770+0000", 2021-01-15T19:10:17.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3300000000000005e-06 2021-01-15T19:10:17.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224779+0000", 2021-01-15T19:10:17.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025161600000000002 2021-01-15T19:10:17.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.225031+0000", 2021-01-15T19:10:17.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:17.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:17.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.225039+0000", 2021-01-15T19:10:17.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013157500000000001 2021-01-15T19:10:17.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226355+0000", 2021-01-15T19:10:17.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154186 2021-01-15T19:10:17.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226509+0000", 2021-01-15T19:10:17.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012755450000000001 2021-01-15T19:10:17.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:17.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.227784+0000", 2021-01-15T19:10:17.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016929759999999999 2021-01-15T19:10:17.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:17.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244714+0000", 2021-01-15T19:10:17.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000165193 2021-01-15T19:10:17.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:17.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244879+0000", 2021-01-15T19:10:17.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029558600000000003 2021-01-15T19:10:17.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245175+0000", 2021-01-15T19:10:17.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02076683 2021-01-15T19:10:17.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.081 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1405 3.10s0 3:0e97918f:::smithi00140323-6:head [append 0~401408] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:17.082 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:17.082 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.537907092, 2021-01-15T19:10:17.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018945849000000001, 2021-01-15T19:10:17.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:17.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:17.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:17.083 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:17.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:17.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1405 2021-01-15T19:10:17.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.084 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:17.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:17.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:17.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:17.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:17.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:17.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013035499999999999 2021-01-15T19:10:17.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:17.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226816+0000", 2021-01-15T19:10:17.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.614e-06 2021-01-15T19:10:17.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226822+0000", 2021-01-15T19:10:17.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1060000000000006e-06 2021-01-15T19:10:17.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226831+0000", 2021-01-15T19:10:17.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7620000000000001e-06 2021-01-15T19:10:17.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226837+0000", 2021-01-15T19:10:17.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061165669999999998 2021-01-15T19:10:17.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.232953+0000", 2021-01-15T19:10:17.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000602875 2021-01-15T19:10:17.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.233556+0000", 2021-01-15T19:10:17.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017456419999999999 2021-01-15T19:10:17.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:17.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.235302+0000", 2021-01-15T19:10:17.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010073057 2021-01-15T19:10:17.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:17.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245375+0000", 2021-01-15T19:10:17.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125055 2021-01-15T19:10:17.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:17.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245500+0000", 2021-01-15T19:10:17.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181600000000001 2021-01-15T19:10:17.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245632+0000", 2021-01-15T19:10:17.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018945849000000001 2021-01-15T19:10:17.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.099 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1406 3.10s0 3:0e97918f:::smithi00140323-6:head [setxattr _header (56) in=7b] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:17.099 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:17.099 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.537731076, 2021-01-15T19:10:17.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027386166999999999, 2021-01-15T19:10:17.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:17.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:17.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:17.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:17.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:17.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1406 2021-01-15T19:10:17.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:17.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:17.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:17.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:17.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:17.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:17.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3294999999999999e-05 2021-01-15T19:10:17.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:17.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226895+0000", 2021-01-15T19:10:17.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6749999999999996e-06 2021-01-15T19:10:17.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226902+0000", 2021-01-15T19:10:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2227e-05 2021-01-15T19:10:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226914+0000", 2021-01-15T19:10:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7149999999999997e-06 2021-01-15T19:10:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226919+0000", 2021-01-15T19:10:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016673698000000001 2021-01-15T19:10:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.243593+0000", 2021-01-15T19:10:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134315 2021-01-15T19:10:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.243727+0000", 2021-01-15T19:10:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00064922199999999997 2021-01-15T19:10:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244376+0000", 2021-01-15T19:10:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023649980000000001 2021-01-15T19:10:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.246741+0000", 2021-01-15T19:10:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073842860000000003 2021-01-15T19:10:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254125+0000", 2021-01-15T19:10:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122736 2021-01-15T19:10:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254248+0000", 2021-01-15T19:10:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027386166999999999 2021-01-15T19:10:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1407 3.10s0 3:0e97918f:::smithi00140323-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e147)", 2021-01-15T19:10:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.537634207, 2021-01-15T19:10:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028972056999999999, 2021-01-15T19:10:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1407 2021-01-15T19:10:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:17.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:17.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0265e-05 2021-01-15T19:10:17.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:17.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226979+0000", 2021-01-15T19:10:17.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1500000000000001e-06 2021-01-15T19:10:17.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:17.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226983+0000", 2021-01-15T19:10:17.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4959999999999999e-06 2021-01-15T19:10:17.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:17.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226991+0000", 2021-01-15T19:10:17.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.185e-06 2021-01-15T19:10:17.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:17.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226994+0000", 2021-01-15T19:10:17.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01880718 2021-01-15T19:10:17.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245801+0000", 2021-01-15T19:10:17.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014826300000000001 2021-01-15T19:10:17.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:17.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245949+0000", 2021-01-15T19:10:17.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083453780000000005 2021-01-15T19:10:17.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:17.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254295+0000", 2021-01-15T19:10:17.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7422999999999994e-05 2021-01-15T19:10:17.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:17.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254372+0000", 2021-01-15T19:10:17.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001558717 2021-01-15T19:10:17.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:17.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:17.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:17.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.255931+0000", 2021-01-15T19:10:17.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028972056999999999 2021-01-15T19:10:17.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:17.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:17.139 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:17.139 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:17.141 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:17.141 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:17.138+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:17.232 INFO:tasks.rados.rados.0.smithi001.stdout:590: done (2 left) 2021-01-15T19:10:17.233 INFO:tasks.rados.rados.0.smithi001.stdout:592: done (1 left) 2021-01-15T19:10:17.233 INFO:tasks.rados.rados.0.smithi001.stdout:594: done (0 left) 2021-01-15T19:10:17.233 INFO:tasks.rados.rados.0.smithi001.stdout:595: rollback oid 13 current snap is 67 2021-01-15T19:10:17.233 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 13 to 51 2021-01-15T19:10:17.233 INFO:tasks.rados.rados.0.smithi001.stdout:596: snap_create 2021-01-15T19:10:17.241 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:17.239+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:17.254 INFO:tasks.rados.rados.0.smithi001.stdout:595: finishing rollback tid 0 to smithi00140323-13 2021-01-15T19:10:17.254 INFO:tasks.rados.rados.0.smithi001.stdout:595: finishing rollback tid 1 to smithi00140323-13 2021-01-15T19:10:17.255 INFO:tasks.rados.rados.0.smithi001.stdout:595: finishing rollback tid 2 to smithi00140323-13 2021-01-15T19:10:17.255 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 272 (ObjNum 165 snap 44 seq_num 165) dirty exists 2021-01-15T19:10:17.329 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:17.330 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:17.330 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:17.330 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:17.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:17.344 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:17.344 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:17.342+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:17.364 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:17.385 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:17.406 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:17.428 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:17.443 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:17.442+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:17.450 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:17.471 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:17.477 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:17.478 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:17.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:17.492 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:17.494 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.494 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.496 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:17.546 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:17.545+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:17.624 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:17.625 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:17.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:17.645 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:17.643+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:17.747 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:17.745+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:17.809 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:17.809 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:17.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:17.848 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:17.846+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:17.949 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:17.948+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:17.991 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:17.992 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:18.001 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:18.085 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:18.084+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:18.151 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:18.150+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:18.174 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:18.174 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:18.184 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:18.198 INFO:tasks.rados.rados.0.smithi001.stdout:595: done (1 left) 2021-01-15T19:10:18.198 INFO:tasks.rados.rados.0.smithi001.stdout:596: done (0 left) 2021-01-15T19:10:18.199 INFO:tasks.rados.rados.0.smithi001.stdout:597: delete oid 39 current snap is 68 2021-01-15T19:10:18.208 INFO:tasks.rados.rados.0.smithi001.stdout:597: done (0 left) 2021-01-15T19:10:18.209 INFO:tasks.rados.rados.0.smithi001.stdout:598: rollback oid 14 current snap is 68 2021-01-15T19:10:18.209 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 14 to 40 2021-01-15T19:10:18.209 INFO:tasks.rados.rados.0.smithi001.stdout:599: rollback oid 11 current snap is 68 2021-01-15T19:10:18.209 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 11 to 62 2021-01-15T19:10:18.209 INFO:tasks.rados.rados.0.smithi001.stdout:600: setattr oid 13 current snap is 68 2021-01-15T19:10:18.210 INFO:tasks.rados.rados.0.smithi001.stdout:601: copy_from oid 42 from oid 36 current snap is 68 2021-01-15T19:10:18.210 INFO:tasks.rados.rados.0.smithi001.stdout:602: delete oid 44 current snap is 68 2021-01-15T19:10:18.218 INFO:tasks.rados.rados.0.smithi001.stdout:598: finishing rollback tid 0 to smithi00140323-14 2021-01-15T19:10:18.224 INFO:tasks.rados.rados.0.smithi001.stdout:602: done (4 left) 2021-01-15T19:10:18.224 INFO:tasks.rados.rados.0.smithi001.stdout:603: setattr oid 47 current snap is 68 2021-01-15T19:10:18.224 INFO:tasks.rados.rados.0.smithi001.stdout:604: copy_from oid 49 from oid 48 current snap is 68 2021-01-15T19:10:18.225 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 25 current snap is 68 2021-01-15T19:10:18.225 INFO:tasks.rados.rados.0.smithi001.stdout:605: seq_num 214 ranges {0=794624,794624=401408} 2021-01-15T19:10:18.226 INFO:tasks.rados.rados.0.smithi001.stdout:605: writing smithi00140323-25 from 0 to 794624 tid 1 2021-01-15T19:10:18.229 INFO:tasks.rados.rados.0.smithi001.stdout:605: writing smithi00140323-25 from 794624 to 1196032 tid 2 2021-01-15T19:10:18.230 INFO:tasks.rados.rados.0.smithi001.stdout:598: finishing rollback tid 1 to smithi00140323-14 2021-01-15T19:10:18.230 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 104 (ObjNum 178933296 snap 22026 seq_num 179299808) dirty dne 2021-01-15T19:10:18.230 INFO:tasks.rados.rados.0.smithi001.stdout:599: finishing rollback tid 0 to smithi00140323-11 2021-01-15T19:10:18.231 INFO:tasks.rados.rados.0.smithi001.stdout:599: finishing rollback tid 1 to smithi00140323-11 2021-01-15T19:10:18.231 INFO:tasks.rados.rados.0.smithi001.stdout:599: finishing rollback tid 2 to smithi00140323-11 2021-01-15T19:10:18.231 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 249 (ObjNum 41 snap 0 seq_num 41) dirty exists 2021-01-15T19:10:18.231 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 159 (ObjNum 179109920 snap 22026 seq_num 179608016) dirty exists 2021-01-15T19:10:18.232 INFO:tasks.rados.rados.0.smithi001.stdout:598: done (6 left) 2021-01-15T19:10:18.232 INFO:tasks.rados.rados.0.smithi001.stdout:599: done (5 left) 2021-01-15T19:10:18.232 INFO:tasks.rados.rados.0.smithi001.stdout:603: done (4 left) 2021-01-15T19:10:18.232 INFO:tasks.rados.rados.0.smithi001.stdout:606: rollback oid 50 current snap is 68 2021-01-15T19:10:18.232 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 50 to 51 2021-01-15T19:10:18.233 INFO:tasks.rados.rados.0.smithi001.stdout:607: delete oid 35 current snap is 68 2021-01-15T19:10:18.237 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 274 (ObjNum 165 snap 44 seq_num 165) dirty exists 2021-01-15T19:10:18.241 INFO:tasks.rados.rados.0.smithi001.stdout:605: finishing write tid 1 to smithi00140323-25 2021-01-15T19:10:18.241 INFO:tasks.rados.rados.0.smithi001.stdout:606: finishing rollback tid 0 to smithi00140323-50 2021-01-15T19:10:18.241 INFO:tasks.rados.rados.0.smithi001.stdout:605: finishing write tid 2 to smithi00140323-25 2021-01-15T19:10:18.242 INFO:tasks.rados.rados.0.smithi001.stdout:605: finishing write tid 3 to smithi00140323-25 2021-01-15T19:10:18.242 INFO:tasks.rados.rados.0.smithi001.stdout:606: finishing rollback tid 1 to smithi00140323-50 2021-01-15T19:10:18.245 INFO:tasks.rados.rados.0.smithi001.stdout:606: finishing rollback tid 2 to smithi00140323-50 2021-01-15T19:10:18.245 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 181 (ObjNum 53 snap 1 seq_num 53) dirty exists 2021-01-15T19:10:18.245 INFO:tasks.rados.rados.0.smithi001.stdout:600: done (5 left) 2021-01-15T19:10:18.245 INFO:tasks.rados.rados.0.smithi001.stdout:606: done (4 left) 2021-01-15T19:10:18.245 INFO:tasks.rados.rados.0.smithi001.stdout:607: done (3 left) 2021-01-15T19:10:18.246 INFO:tasks.rados.rados.0.smithi001.stdout:608: read oid 38 snap -1 2021-01-15T19:10:18.246 INFO:tasks.rados.rados.0.smithi001.stdout:608: expect (ObjNum 207 snap 65 seq_num 207) 2021-01-15T19:10:18.246 INFO:tasks.rados.rados.0.smithi001.stdout:609: copy_from oid 40 from oid 1 current snap is 68 2021-01-15T19:10:18.246 INFO:tasks.rados.rados.0.smithi001.stdout:610: delete oid 47 current snap is 68 2021-01-15T19:10:18.248 INFO:tasks.rados.rados.0.smithi001.stdout:605: finishing write tid 4 to smithi00140323-25 2021-01-15T19:10:18.249 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 25 v 277 (ObjNum 214 snap 68 seq_num 214) dirty exists 2021-01-15T19:10:18.249 INFO:tasks.rados.rados.0.smithi001.stdout:605: left oid 25 (ObjNum 214 snap 68 seq_num 214) 2021-01-15T19:10:18.251 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:18.250+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:18.254 INFO:tasks.rados.rados.0.smithi001.stdout:605: done (5 left) 2021-01-15T19:10:18.255 INFO:tasks.rados.rados.0.smithi001.stdout:610: done (4 left) 2021-01-15T19:10:18.255 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 8 current snap is 68 2021-01-15T19:10:18.255 INFO:tasks.rados.rados.0.smithi001.stdout:611: seq_num 217 ranges {4128768=442368,4571136=696320,5267456=106496} 2021-01-15T19:10:18.281 INFO:tasks.rados.rados.0.smithi001.stdout:611: writing smithi00140323-8 from 4128768 to 4571136 tid 1 2021-01-15T19:10:18.286 INFO:tasks.rados.rados.0.smithi001.stdout:611: writing smithi00140323-8 from 4571136 to 5267456 tid 2 2021-01-15T19:10:18.287 INFO:tasks.rados.rados.0.smithi001.stdout:611: writing smithi00140323-8 from 5267456 to 5373952 tid 3 2021-01-15T19:10:18.287 INFO:tasks.rados.rados.0.smithi001.stdout:612: read oid 5 snap 54 2021-01-15T19:10:18.288 INFO:tasks.rados.rados.0.smithi001.stdout:612: expect (ObjNum 163 snap 43 seq_num 163) 2021-01-15T19:10:18.288 INFO:tasks.rados.rados.0.smithi001.stdout:613: snap_create 2021-01-15T19:10:18.288 INFO:tasks.rados.rados.0.smithi001.stdout:609: finishing copy_from to smithi00140323-40 2021-01-15T19:10:18.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 40 v 146 (ObjNum 191 snap 59 seq_num 191) dirty exists 2021-01-15T19:10:18.288 INFO:tasks.rados.rados.0.smithi001.stdout:609: finishing copy_from racing read to smithi00140323-40 2021-01-15T19:10:18.289 INFO:tasks.rados.rados.0.smithi001.stdout:601: finishing copy_from to smithi00140323-42 2021-01-15T19:10:18.289 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 133 (ObjNum 180 snap 56 seq_num 180) dirty exists 2021-01-15T19:10:18.289 INFO:tasks.rados.rados.0.smithi001.stdout:601: finishing copy_from racing read to smithi00140323-42 2021-01-15T19:10:18.289 INFO:tasks.rados.rados.0.smithi001.stdout:608: expect (ObjNum 207 snap 65 seq_num 207) 2021-01-15T19:10:18.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:10:18.352 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:18.350+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:18.352 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:18.352 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:18.362 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:18.452 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:18.451+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:18.527 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:18.528 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:18.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:18.553 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:18.552+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:18.600 INFO:tasks.rados.rados.0.smithi001.stdout:604: finishing copy_from to smithi00140323-49 2021-01-15T19:10:18.601 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 148 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:10:18.601 INFO:tasks.rados.rados.0.smithi001.stdout:604: finishing copy_from racing read to smithi00140323-49 2021-01-15T19:10:18.601 INFO:tasks.rados.rados.0.smithi001.stdout:612: expect (ObjNum 163 snap 43 seq_num 163) 2021-01-15T19:10:18.630 INFO:tasks.rados.rados.0.smithi001.stdout:611: finishing write tid 1 to smithi00140323-8 2021-01-15T19:10:18.630 INFO:tasks.rados.rados.0.smithi001.stdout:611: finishing write tid 2 to smithi00140323-8 2021-01-15T19:10:18.630 INFO:tasks.rados.rados.0.smithi001.stdout:611: finishing write tid 3 to smithi00140323-8 2021-01-15T19:10:18.631 INFO:tasks.rados.rados.0.smithi001.stdout:611: finishing write tid 4 to smithi00140323-8 2021-01-15T19:10:18.631 INFO:tasks.rados.rados.0.smithi001.stdout:611: finishing write tid 5 to smithi00140323-8 2021-01-15T19:10:18.631 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 8 v 186 (ObjNum 217 snap 68 seq_num 217) dirty exists 2021-01-15T19:10:18.631 INFO:tasks.rados.rados.0.smithi001.stdout:611: left oid 8 (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:18.654 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:18.652+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:18.661 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:18.662 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:18.671 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:18.756 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:18.754+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:18.837 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:18.838 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:18.847 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:18.855 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:18.854+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:18.955 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:18.953+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:19.012 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:19.012 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:19.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:19.056 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:19.055+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:19.157 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:19.155+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:19.188 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:19.188 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:19.196 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:19.247 INFO:tasks.rados.rados.0.smithi001.stdout:601: done (6 left) 2021-01-15T19:10:19.248 INFO:tasks.rados.rados.0.smithi001.stdout:604: done (5 left) 2021-01-15T19:10:19.248 INFO:tasks.rados.rados.0.smithi001.stdout:608: done (4 left) 2021-01-15T19:10:19.248 INFO:tasks.rados.rados.0.smithi001.stdout:609: done (3 left) 2021-01-15T19:10:19.248 INFO:tasks.rados.rados.0.smithi001.stdout:611: done (2 left) 2021-01-15T19:10:19.249 INFO:tasks.rados.rados.0.smithi001.stdout:612: done (1 left) 2021-01-15T19:10:19.249 INFO:tasks.rados.rados.0.smithi001.stdout:613: done (0 left) 2021-01-15T19:10:19.249 INFO:tasks.rados.rados.0.smithi001.stdout:614: read oid 8 snap -1 2021-01-15T19:10:19.249 INFO:tasks.rados.rados.0.smithi001.stdout:614: expect (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:19.249 INFO:tasks.rados.rados.0.smithi001.stdout:615: snap_create 2021-01-15T19:10:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:10:19.257 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:19.256+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:19.283 INFO:tasks.rados.rados.0.smithi001.stdout:614: expect (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:19.359 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:19.357+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:19.368 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:19.368 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:19.377 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:19.460 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:19.458+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:19.505 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:19.542 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:19.542 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:19.551 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:19.595 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:19.593+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:19.653 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:19.661 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:19.660+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:19.673 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:19.675 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:19.675 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:19.685 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:19.694 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:19.714 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:19.736 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:19.757 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:19.770 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:19.768+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:19.778 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:19.798 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.800 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.800 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.800 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.800 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.800 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.801 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:19.852 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:19.853 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:19.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:19.864 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:19.864+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:19.965 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:19.963+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:20.027 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:20.028 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:20.029 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:20.037 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:20.066 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:20.065+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:20.167 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:20.165+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:20.201 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:20.202 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:20.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:20.202 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:20.202 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:20.212 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:20.245 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:20.245 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:10:20.246 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2021-01-15T19:10:20.246 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:10:20.256 INFO:tasks.rados.rados.0.smithi001.stdout:614: done (1 left) 2021-01-15T19:10:20.256 INFO:tasks.rados.rados.0.smithi001.stdout:615: done (0 left) 2021-01-15T19:10:20.256 INFO:tasks.rados.rados.0.smithi001.stdout:616: snap_remove snap 51 2021-01-15T19:10:20.299 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:20.298+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:20.352 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:20.352 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:20.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:20.404 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:20.403+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:20.469 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:20.468+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:20.535 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:20.536 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:20.537 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:20.537 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:20.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:10:20.576 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:20.572+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:10:20.587 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2021-01-15T19:10:20.252156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":167346176,"num_objects":137,"num_object_clones":90,"num_object_copies":409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":10,"num_read":465,"num_read_kb":439996,"num_write":754,"num_write_kb":176665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":248659968,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2742,"ondisk_log_size":2742,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":453580,"kb_used_data":273052,"kb_used_omap":0,"kb_used_meta":180480,"kb_avail":565777460,"statfs":{"total":579820584960,"available":579356119040,"internally_reserved":0,"allocated":279605248,"data_stored":506561408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":184811520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18350080,"num_objects":-19,"num_object_clones":-18,"num_object_copies":-57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-19,"num_whiteouts":-1,"num_read":5,"num_read_kb":13023,"num_write":3,"num_write_kb":1096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028331"},"pg_stats":[{"pgid":"3.19","version":"143'59","reported_seq":"246","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216599+0000","last_change":"2021-01-15T19:09:27.212298+0000","last_active":"2021-01-15T19:10:14.216599+0000","last_peered":"2021-01-15T19:10:14.216599+0000","last_clean":"2021-01-15T19:10:14.216599+0000","last_became_active":"2021-01-15T19:08:46.196966+0000","last_became_peered":"2021-01-15T19:08:46.196966+0000","last_unstale":"2021-01-15T19:10:14.216599+0000","last_undegraded":"2021-01-15T19:10:14.216599+0000","last_fullsized":"2021-01-15T19:10:14.216599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'51","last_scrub_stamp":"2021-01-15T19:09:27.212108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:27.212108+0000","log_size":59,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":21,"stat_sum":{"num_bytes":6373376,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":1048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"146'72","reported_seq":"386","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.243203+0000","last_change":"2021-01-15T19:10:15.244647+0000","last_active":"2021-01-15T19:10:15.243203+0000","last_peered":"2021-01-15T19:10:15.243203+0000","last_clean":"2021-01-15T19:10:15.243203+0000","last_became_active":"2021-01-15T19:09:51.207469+0000","last_became_peered":"2021-01-15T19:09:51.207469+0000","last_unstale":"2021-01-15T19:10:15.243203+0000","last_undegraded":"2021-01-15T19:10:15.243203+0000","last_fullsized":"2021-01-15T19:10:15.243203+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"146'72","last_scrub_stamp":"2021-01-15T19:10:14.380333+0000","last_deep_scrub":"51'34","last_deep_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_clean_scrub_stamp":"2021-01-15T19:10:14.380333+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1785856,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2235,"num_write":13,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":3047424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"3.17","version":"50'30","reported_seq":"179","reported_epoch":"148","state":"active+clean","last_fresh":"2021-01-15T19:10:16.271742+0000","last_change":"2021-01-15T19:10:16.272174+0000","last_active":"2021-01-15T19:10:16.271742+0000","last_peered":"2021-01-15T19:10:16.271742+0000","last_clean":"2021-01-15T19:10:16.271742+0000","last_became_active":"2021-01-15T19:09:21.207252+0000","last_became_peered":"2021-01-15T19:09:21.207252+0000","last_unstale":"2021-01-15T19:10:16.271742+0000","last_undegraded":"2021-01-15T19:10:16.271742+0000","last_fullsized":"2021-01-15T19:10:16.271742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'30","last_scrub_stamp":"2021-01-15T19:09:25.976466+0000","last_deep_scrub":"50'30","last_deep_scrub_stamp":"2021-01-15T19:09:04.507336+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.976466+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"},{"start":"71","length":"1"}]},{"pgid":"3.16","version":"146'91","reported_seq":"389","reported_epoch":"147","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:15.235382+0000","last_change":"2021-01-15T19:10:15.235468+0000","last_active":"2021-01-15T19:10:15.235382+0000","last_peered":"2021-01-15T19:10:15.235382+0000","last_clean":"2021-01-15T19:10:15.235382+0000","last_became_active":"2021-01-15T19:09:15.343356+0000","last_became_peered":"2021-01-15T19:09:15.343356+0000","last_unstale":"2021-01-15T19:10:15.235382+0000","last_undegraded":"2021-01-15T19:10:15.235382+0000","last_fullsized":"2021-01-15T19:10:15.235382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":5,"last_scrub":"58'32","last_scrub_stamp":"2021-01-15T19:08:51.873289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.873289+0000","log_size":91,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":7331840,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":7277,"num_write":17,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2621440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"3.15","version":"148'70","reported_seq":"369","reported_epoch":"148","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:16.252702+0000","last_change":"2021-01-15T19:10:16.252702+0000","last_active":"2021-01-15T19:10:16.252702+0000","last_peered":"2021-01-15T19:10:16.252702+0000","last_clean":"2021-01-15T19:10:16.252702+0000","last_became_active":"2021-01-15T19:09:51.209994+0000","last_became_peered":"2021-01-15T19:09:51.209994+0000","last_unstale":"2021-01-15T19:10:16.252702+0000","last_undegraded":"2021-01-15T19:10:16.252702+0000","last_fullsized":"2021-01-15T19:10:16.252702+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'32","last_scrub_stamp":"2021-01-15T19:09:03.911966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:03.911966+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":5152768,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2145,"num_write":25,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5455872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"3.14","version":"146'92","reported_seq":"303","reported_epoch":"147","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:15.243980+0000","last_change":"2021-01-15T19:10:15.243980+0000","last_active":"2021-01-15T19:10:15.243980+0000","last_peered":"2021-01-15T19:10:15.243980+0000","last_clean":"2021-01-15T19:10:15.243980+0000","last_became_active":"2021-01-15T19:09:51.208908+0000","last_became_peered":"2021-01-15T19:09:51.208908+0000","last_unstale":"2021-01-15T19:10:15.243980+0000","last_undegraded":"2021-01-15T19:10:15.243980+0000","last_fullsized":"2021-01-15T19:10:15.243980+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":92,"ondisk_log_size":92,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":6184960,"num_objects":3,"num_object_clones":3,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":4024,"num_write":19,"num_write_kb":11128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":21037056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":"294","reported_epoch":"148","state":"active+clean","last_fresh":"2021-01-15T19:10:16.255658+0000","last_change":"2021-01-15T19:10:16.256165+0000","last_active":"2021-01-15T19:10:16.255658+0000","last_peered":"2021-01-15T19:10:16.255658+0000","last_clean":"2021-01-15T19:10:16.255658+0000","last_became_active":"2021-01-15T19:08:43.203849+0000","last_became_peered":"2021-01-15T19:08:43.203849+0000","last_unstale":"2021-01-15T19:10:16.255658+0000","last_undegraded":"2021-01-15T19:10:16.255658+0000","last_fullsized":"2021-01-15T19:10:16.255658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.175599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"},{"start":"71","length":"1"}]},{"pgid":"3.12","version":"147'130","reported_seq":"504","reported_epoch":"147","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:15.245767+0000","last_change":"2021-01-15T19:10:15.240080+0000","last_active":"2021-01-15T19:10:15.245767+0000","last_peered":"2021-01-15T19:10:15.245767+0000","last_clean":"2021-01-15T19:10:15.245767+0000","last_became_active":"2021-01-15T19:09:51.213346+0000","last_became_peered":"2021-01-15T19:09:51.213346+0000","last_unstale":"2021-01-15T19:10:15.245767+0000","last_undegraded":"2021-01-15T19:10:15.245767+0000","last_fullsized":"2021-01-15T19:10:15.245767+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'42","last_scrub_stamp":"2021-01-15T19:09:02.894120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.894120+0000","log_size":130,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":11067392,"num_objects":6,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":18,"num_read_kb":17863,"num_write":31,"num_write_kb":6456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":57647104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"1.4","version":"14'2","reported_seq":"101","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.226270+0000","last_change":"2021-01-15T19:08:53.068011+0000","last_active":"2021-01-15T19:10:15.226270+0000","last_peered":"2021-01-15T19:10:15.226270+0000","last_clean":"2021-01-15T19:10:15.226270+0000","last_became_active":"2021-01-15T19:08:52.197215+0000","last_became_peered":"2021-01-15T19:08:52.197215+0000","last_unstale":"2021-01-15T19:10:15.226270+0000","last_undegraded":"2021-01-15T19:10:15.226270+0000","last_fullsized":"2021-01-15T19:10:15.226270+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"147'140","reported_seq":"495","reported_epoch":"147","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:15.253175+0000","last_change":"2021-01-15T19:10:15.246174+0000","last_active":"2021-01-15T19:10:15.253175+0000","last_peered":"2021-01-15T19:10:15.253175+0000","last_clean":"2021-01-15T19:10:15.253175+0000","last_became_active":"2021-01-15T19:08:41.227163+0000","last_became_peered":"2021-01-15T19:08:41.227163+0000","last_unstale":"2021-01-15T19:10:15.253175+0000","last_undegraded":"2021-01-15T19:10:15.253175+0000","last_fullsized":"2021-01-15T19:10:15.253175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'62","last_scrub_stamp":"2021-01-15T19:09:14.332675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:14.332675+0000","log_size":140,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":6447104,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":9,"num_read_kb":8045,"num_write":33,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":"127","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.231688+0000","last_change":"2021-01-15T19:09:48.209380+0000","last_active":"2021-01-15T19:10:15.231688+0000","last_peered":"2021-01-15T19:10:15.231688+0000","last_clean":"2021-01-15T19:10:15.231688+0000","last_became_active":"2021-01-15T19:09:48.208922+0000","last_became_peered":"2021-01-15T19:09:48.208922+0000","last_unstale":"2021-01-15T19:10:15.231688+0000","last_undegraded":"2021-01-15T19:10:15.231688+0000","last_fullsized":"2021-01-15T19:10:15.231688+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.960545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.960545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"64'3","reported_seq":"137","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216656+0000","last_change":"2021-01-15T19:09:51.205925+0000","last_active":"2021-01-15T19:10:14.216656+0000","last_peered":"2021-01-15T19:10:14.216656+0000","last_clean":"2021-01-15T19:10:14.216656+0000","last_became_active":"2021-01-15T19:09:51.202906+0000","last_became_peered":"2021-01-15T19:09:51.202906+0000","last_unstale":"2021-01-15T19:10:14.216656+0000","last_undegraded":"2021-01-15T19:10:14.216656+0000","last_fullsized":"2021-01-15T19:10:14.216656+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":3,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.906630+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"154","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216741+0000","last_change":"2021-01-15T19:09:54.103363+0000","last_active":"2021-01-15T19:10:14.216741+0000","last_peered":"2021-01-15T19:10:14.216741+0000","last_clean":"2021-01-15T19:10:14.216741+0000","last_became_active":"2021-01-15T19:08:35.206099+0000","last_became_peered":"2021-01-15T19:08:35.206099+0000","last_unstale":"2021-01-15T19:10:14.216741+0000","last_undegraded":"2021-01-15T19:10:14.216741+0000","last_fullsized":"2021-01-15T19:10:14.216741+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.103280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.103280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"146'65","reported_seq":"477","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.245575+0000","last_change":"2021-01-15T19:10:15.246108+0000","last_active":"2021-01-15T19:10:15.245575+0000","last_peered":"2021-01-15T19:10:15.245575+0000","last_clean":"2021-01-15T19:10:15.245575+0000","last_became_active":"2021-01-15T19:09:51.204796+0000","last_became_peered":"2021-01-15T19:09:51.204796+0000","last_unstale":"2021-01-15T19:10:15.245575+0000","last_undegraded":"2021-01-15T19:10:15.245575+0000","last_fullsized":"2021-01-15T19:10:15.245575+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'37","last_scrub_stamp":"2021-01-15T19:08:54.947352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:54.947352+0000","log_size":65,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":802816,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":36508,"num_write":6,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":8904704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"1.6","version":"14'2","reported_seq":"83","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.226437+0000","last_change":"2021-01-15T19:09:51.204380+0000","last_active":"2021-01-15T19:10:15.226437+0000","last_peered":"2021-01-15T19:10:15.226437+0000","last_clean":"2021-01-15T19:10:15.226437+0000","last_became_active":"2021-01-15T19:09:51.203386+0000","last_became_peered":"2021-01-15T19:09:51.203386+0000","last_unstale":"2021-01-15T19:10:15.226437+0000","last_undegraded":"2021-01-15T19:10:15.226437+0000","last_fullsized":"2021-01-15T19:10:15.226437+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:15.692639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:15.692639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"147'79","reported_seq":"485","reported_epoch":"147","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:15.231569+0000","last_change":"2021-01-15T19:10:15.231569+0000","last_active":"2021-01-15T19:10:15.231569+0000","last_peered":"2021-01-15T19:10:15.231569+0000","last_clean":"2021-01-15T19:10:15.231569+0000","last_became_active":"2021-01-15T19:09:51.206304+0000","last_became_peered":"2021-01-15T19:09:51.206304+0000","last_unstale":"2021-01-15T19:10:15.231569+0000","last_undegraded":"2021-01-15T19:10:15.231569+0000","last_fullsized":"2021-01-15T19:10:15.231569+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'69","last_scrub_stamp":"2021-01-15T19:10:01.942460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.942460+0000","log_size":79,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":5079040,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":7,"num_read_kb":1825,"num_write":4,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"147","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.220460+0000","last_change":"2021-01-15T19:08:55.065884+0000","last_active":"2021-01-15T19:10:15.220460+0000","last_peered":"2021-01-15T19:10:15.220460+0000","last_clean":"2021-01-15T19:10:15.220460+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:10:15.220460+0000","last_undegraded":"2021-01-15T19:10:15.220460+0000","last_fullsized":"2021-01-15T19:10:15.220460+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_clean_scrub_stamp":"2021-01-15T19:08:55.065810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"164","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216579+0000","last_change":"2021-01-15T19:09:28.194990+0000","last_active":"2021-01-15T19:10:14.216579+0000","last_peered":"2021-01-15T19:10:14.216579+0000","last_clean":"2021-01-15T19:10:14.216579+0000","last_became_active":"2021-01-15T19:08:52.202463+0000","last_became_peered":"2021-01-15T19:08:52.202463+0000","last_unstale":"2021-01-15T19:10:14.216579+0000","last_undegraded":"2021-01-15T19:10:14.216579+0000","last_fullsized":"2021-01-15T19:10:14.216579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:28.194883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.194883+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"148'101","reported_seq":"470","reported_epoch":"148","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:16.244865+0000","last_change":"2021-01-15T19:10:16.244865+0000","last_active":"2021-01-15T19:10:16.244865+0000","last_peered":"2021-01-15T19:10:16.244865+0000","last_clean":"2021-01-15T19:10:16.244865+0000","last_became_active":"2021-01-15T19:09:51.207663+0000","last_became_peered":"2021-01-15T19:09:51.207663+0000","last_unstale":"2021-01-15T19:10:16.244865+0000","last_undegraded":"2021-01-15T19:10:16.244865+0000","last_fullsized":"2021-01-15T19:10:16.244865+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'47","last_scrub_stamp":"2021-01-15T19:08:53.015941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.015941+0000","log_size":101,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":2482176,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":39,"num_write_kb":6240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":7340032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"172","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.219699+0000","last_change":"2021-01-15T19:09:53.454822+0000","last_active":"2021-01-15T19:10:15.219699+0000","last_peered":"2021-01-15T19:10:15.219699+0000","last_clean":"2021-01-15T19:10:15.219699+0000","last_became_active":"2021-01-15T19:09:48.201031+0000","last_became_peered":"2021-01-15T19:09:48.201031+0000","last_unstale":"2021-01-15T19:10:15.219699+0000","last_undegraded":"2021-01-15T19:10:15.219699+0000","last_fullsized":"2021-01-15T19:10:15.219699+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:53.454739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.454739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"382","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.244763+0000","last_change":"2021-01-15T19:10:15.245417+0000","last_active":"2021-01-15T19:10:15.244763+0000","last_peered":"2021-01-15T19:10:15.244763+0000","last_clean":"2021-01-15T19:10:15.244763+0000","last_became_active":"2021-01-15T19:08:41.234856+0000","last_became_peered":"2021-01-15T19:08:41.234856+0000","last_unstale":"2021-01-15T19:10:15.244763+0000","last_undegraded":"2021-01-15T19:10:15.244763+0000","last_fullsized":"2021-01-15T19:10:15.244763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'18","last_scrub_stamp":"2021-01-15T19:09:16.308952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.308952+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"1.0","version":"14'2","reported_seq":"171","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.223239+0000","last_change":"2021-01-15T19:09:21.202209+0000","last_active":"2021-01-15T19:10:14.223239+0000","last_peered":"2021-01-15T19:10:14.223239+0000","last_clean":"2021-01-15T19:10:14.223239+0000","last_became_active":"2021-01-15T19:09:21.201680+0000","last_became_peered":"2021-01-15T19:09:21.201680+0000","last_unstale":"2021-01-15T19:10:14.223239+0000","last_undegraded":"2021-01-15T19:10:14.223239+0000","last_fullsized":"2021-01-15T19:10:14.223239+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:52.730150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.730150+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"147","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.220135+0000","last_change":"2021-01-15T19:09:04.092968+0000","last_active":"2021-01-15T19:10:15.220135+0000","last_peered":"2021-01-15T19:10:15.220135+0000","last_clean":"2021-01-15T19:10:15.220135+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:10:15.220135+0000","last_undegraded":"2021-01-15T19:10:15.220135+0000","last_fullsized":"2021-01-15T19:10:15.220135+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:04.092902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:04.092902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"107","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.230293+0000","last_change":"2021-01-15T19:09:16.945459+0000","last_active":"2021-01-15T19:10:15.230293+0000","last_peered":"2021-01-15T19:10:15.230293+0000","last_clean":"2021-01-15T19:10:15.230293+0000","last_became_active":"2021-01-15T19:08:52.205001+0000","last_became_peered":"2021-01-15T19:08:52.205001+0000","last_unstale":"2021-01-15T19:10:15.230293+0000","last_undegraded":"2021-01-15T19:10:15.230293+0000","last_fullsized":"2021-01-15T19:10:15.230293+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:16.945387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.945387+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"146'82","reported_seq":"428","reported_epoch":"147","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:15.243088+0000","last_change":"2021-01-15T19:10:15.243173+0000","last_active":"2021-01-15T19:10:15.243088+0000","last_peered":"2021-01-15T19:10:15.243088+0000","last_clean":"2021-01-15T19:10:15.243088+0000","last_became_active":"2021-01-15T19:09:21.202949+0000","last_became_peered":"2021-01-15T19:09:21.202949+0000","last_unstale":"2021-01-15T19:10:15.243088+0000","last_undegraded":"2021-01-15T19:10:15.243088+0000","last_fullsized":"2021-01-15T19:10:15.243088+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'68","last_scrub_stamp":"2021-01-15T19:09:29.177377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.177377+0000","log_size":82,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":3506176,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2353,"num_write":2,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"171","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.226369+0000","last_change":"2021-01-15T19:10:05.030900+0000","last_active":"2021-01-15T19:10:15.226369+0000","last_peered":"2021-01-15T19:10:15.226369+0000","last_clean":"2021-01-15T19:10:15.226369+0000","last_became_active":"2021-01-15T19:09:18.204537+0000","last_became_peered":"2021-01-15T19:09:18.204537+0000","last_unstale":"2021-01-15T19:10:15.226369+0000","last_undegraded":"2021-01-15T19:10:15.226369+0000","last_fullsized":"2021-01-15T19:10:15.226369+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_clean_scrub_stamp":"2021-01-15T19:10:05.030822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"86","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216761+0000","last_change":"2021-01-15T19:09:55.119804+0000","last_active":"2021-01-15T19:10:14.216761+0000","last_peered":"2021-01-15T19:10:14.216761+0000","last_clean":"2021-01-15T19:10:14.216761+0000","last_became_active":"2021-01-15T19:09:51.205453+0000","last_became_peered":"2021-01-15T19:09:51.205453+0000","last_unstale":"2021-01-15T19:10:14.216761+0000","last_undegraded":"2021-01-15T19:10:14.216761+0000","last_fullsized":"2021-01-15T19:10:14.216761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:55.119702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:55.119702+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"354","reported_epoch":"147","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:15.236737+0000","last_change":"2021-01-15T19:10:15.236737+0000","last_active":"2021-01-15T19:10:15.236737+0000","last_peered":"2021-01-15T19:10:15.236737+0000","last_clean":"2021-01-15T19:10:15.236737+0000","last_became_active":"2021-01-15T19:08:41.228264+0000","last_became_peered":"2021-01-15T19:08:41.228264+0000","last_unstale":"2021-01-15T19:10:15.236737+0000","last_undegraded":"2021-01-15T19:10:15.236737+0000","last_fullsized":"2021-01-15T19:10:15.236737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:52.822027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:52.822027+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"109","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.223835+0000","last_change":"2021-01-15T19:09:54.187587+0000","last_active":"2021-01-15T19:10:14.223835+0000","last_peered":"2021-01-15T19:10:14.223835+0000","last_clean":"2021-01-15T19:10:14.223835+0000","last_became_active":"2021-01-15T19:09:48.206022+0000","last_became_peered":"2021-01-15T19:09:48.206022+0000","last_unstale":"2021-01-15T19:10:14.223835+0000","last_undegraded":"2021-01-15T19:10:14.223835+0000","last_fullsized":"2021-01-15T19:10:14.223835+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.187515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.187515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"190","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.224039+0000","last_change":"2021-01-15T19:09:51.213883+0000","last_active":"2021-01-15T19:10:14.224039+0000","last_peered":"2021-01-15T19:10:14.224039+0000","last_clean":"2021-01-15T19:10:14.224039+0000","last_became_active":"2021-01-15T19:09:51.213470+0000","last_became_peered":"2021-01-15T19:09:51.213470+0000","last_unstale":"2021-01-15T19:10:14.224039+0000","last_undegraded":"2021-01-15T19:10:14.224039+0000","last_fullsized":"2021-01-15T19:10:14.224039+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"147'110","reported_seq":"522","reported_epoch":"147","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:15.241962+0000","last_change":"2021-01-15T19:10:15.235601+0000","last_active":"2021-01-15T19:10:15.241962+0000","last_peered":"2021-01-15T19:10:15.241962+0000","last_clean":"2021-01-15T19:10:15.241962+0000","last_became_active":"2021-01-15T19:09:51.202675+0000","last_became_peered":"2021-01-15T19:09:51.202675+0000","last_unstale":"2021-01-15T19:10:15.241962+0000","last_undegraded":"2021-01-15T19:10:15.241962+0000","last_fullsized":"2021-01-15T19:10:15.241962+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'90","last_scrub_stamp":"2021-01-15T19:10:02.902935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.902935+0000","log_size":110,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":7380992,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":19,"num_read_kb":25999,"num_write":15,"num_write_kb":7208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12623872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"110","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.217756+0000","last_change":"2021-01-15T19:09:49.884000+0000","last_active":"2021-01-15T19:10:14.217756+0000","last_peered":"2021-01-15T19:10:14.217756+0000","last_clean":"2021-01-15T19:10:14.217756+0000","last_became_active":"2021-01-15T19:09:48.203471+0000","last_became_peered":"2021-01-15T19:09:48.203471+0000","last_unstale":"2021-01-15T19:10:14.217756+0000","last_undegraded":"2021-01-15T19:10:14.217756+0000","last_fullsized":"2021-01-15T19:10:14.217756+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:49.883930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:49.883930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.a","version":"14'2","reported_seq":"194","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.223945+0000","last_change":"2021-01-15T19:10:07.176493+0000","last_active":"2021-01-15T19:10:14.223945+0000","last_peered":"2021-01-15T19:10:14.223945+0000","last_clean":"2021-01-15T19:10:14.223945+0000","last_became_active":"2021-01-15T19:09:51.212693+0000","last_became_peered":"2021-01-15T19:09:51.212693+0000","last_unstale":"2021-01-15T19:10:14.223945+0000","last_undegraded":"2021-01-15T19:10:14.223945+0000","last_fullsized":"2021-01-15T19:10:14.223945+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_clean_scrub_stamp":"2021-01-15T19:10:07.176419+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"147'256","reported_seq":"593","reported_epoch":"147","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:15.239253+0000","last_change":"2021-01-15T19:10:15.231428+0000","last_active":"2021-01-15T19:10:15.239253+0000","last_peered":"2021-01-15T19:10:15.239253+0000","last_clean":"2021-01-15T19:10:15.239253+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:10:15.239253+0000","last_undegraded":"2021-01-15T19:10:15.239253+0000","last_fullsized":"2021-01-15T19:10:15.239253+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'109","last_scrub_stamp":"2021-01-15T19:09:15.296470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:15.296470+0000","log_size":256,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":19726336,"num_objects":16,"num_object_clones":12,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":13,"num_read_kb":19615,"num_write":43,"num_write_kb":14256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":"111","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216444+0000","last_change":"2021-01-15T19:08:50.981704+0000","last_active":"2021-01-15T19:10:14.216444+0000","last_peered":"2021-01-15T19:10:14.216444+0000","last_clean":"2021-01-15T19:10:14.216444+0000","last_became_active":"2021-01-15T19:08:37.215228+0000","last_became_peered":"2021-01-15T19:08:37.215228+0000","last_unstale":"2021-01-15T19:10:14.216444+0000","last_undegraded":"2021-01-15T19:10:14.216444+0000","last_fullsized":"2021-01-15T19:10:14.216444+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:50.981612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:50.981612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"148'177","reported_seq":"557","reported_epoch":"148","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:16.272263+0000","last_change":"2021-01-15T19:10:16.272263+0000","last_active":"2021-01-15T19:10:16.272263+0000","last_peered":"2021-01-15T19:10:16.272263+0000","last_clean":"2021-01-15T19:10:16.272263+0000","last_became_active":"2021-01-15T19:09:15.341093+0000","last_became_peered":"2021-01-15T19:09:15.341093+0000","last_unstale":"2021-01-15T19:10:16.272263+0000","last_undegraded":"2021-01-15T19:10:16.272263+0000","last_fullsized":"2021-01-15T19:10:16.272263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"78'61","last_scrub_stamp":"2021-01-15T19:09:07.109680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:07.109680+0000","log_size":177,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":14843904,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":24,"num_read_kb":30426,"num_write":75,"num_write_kb":10296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"112","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.230002+0000","last_change":"2021-01-15T19:08:51.826450+0000","last_active":"2021-01-15T19:10:15.230002+0000","last_peered":"2021-01-15T19:10:15.230002+0000","last_clean":"2021-01-15T19:10:15.230002+0000","last_became_active":"2021-01-15T19:08:37.216976+0000","last_became_peered":"2021-01-15T19:08:37.216976+0000","last_unstale":"2021-01-15T19:10:15.230002+0000","last_undegraded":"2021-01-15T19:10:15.230002+0000","last_fullsized":"2021-01-15T19:10:15.230002+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:51.826360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:51.826360+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.8","version":"14'2","reported_seq":"170","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.223344+0000","last_change":"2021-01-15T19:09:06.583849+0000","last_active":"2021-01-15T19:10:14.223344+0000","last_peered":"2021-01-15T19:10:14.223344+0000","last_clean":"2021-01-15T19:10:14.223344+0000","last_became_active":"2021-01-15T19:08:52.204479+0000","last_became_peered":"2021-01-15T19:08:52.204479+0000","last_unstale":"2021-01-15T19:10:14.223344+0000","last_undegraded":"2021-01-15T19:10:14.223344+0000","last_fullsized":"2021-01-15T19:10:14.223344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:06.583774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:06.583774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"148'172","reported_seq":"537","reported_epoch":"148","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:16.255574+0000","last_change":"2021-01-15T19:10:16.255574+0000","last_active":"2021-01-15T19:10:16.255574+0000","last_peered":"2021-01-15T19:10:16.255574+0000","last_clean":"2021-01-15T19:10:16.255574+0000","last_became_active":"2021-01-15T19:08:10.704005+0000","last_became_peered":"2021-01-15T19:08:10.704005+0000","last_unstale":"2021-01-15T19:10:16.255574+0000","last_undegraded":"2021-01-15T19:10:16.255574+0000","last_fullsized":"2021-01-15T19:10:16.255574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'120","last_scrub_stamp":"2021-01-15T19:09:30.324786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.324786+0000","log_size":172,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":9748480,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":53,"num_read_kb":44420,"num_write":70,"num_write_kb":17216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.b","version":"0'0","reported_seq":"146","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.220292+0000","last_change":"2021-01-15T19:09:05.065723+0000","last_active":"2021-01-15T19:10:15.220292+0000","last_peered":"2021-01-15T19:10:15.220292+0000","last_clean":"2021-01-15T19:10:15.220292+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:10:15.220292+0000","last_undegraded":"2021-01-15T19:10:15.220292+0000","last_fullsized":"2021-01-15T19:10:15.220292+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:05.065643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:05.065643+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"14'2","reported_seq":"174","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.224116+0000","last_change":"2021-01-15T19:09:18.403315+0000","last_active":"2021-01-15T19:10:14.224116+0000","last_peered":"2021-01-15T19:10:14.224116+0000","last_clean":"2021-01-15T19:10:14.224116+0000","last_became_active":"2021-01-15T19:08:52.203041+0000","last_became_peered":"2021-01-15T19:08:52.203041+0000","last_unstale":"2021-01-15T19:10:14.224116+0000","last_undegraded":"2021-01-15T19:10:14.224116+0000","last_fullsized":"2021-01-15T19:10:14.224116+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:18.403239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.403239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"147'56","reported_seq":"468","reported_epoch":"148","state":"active+clean","last_fresh":"2021-01-15T19:10:16.271398+0000","last_change":"2021-01-15T19:10:16.271674+0000","last_active":"2021-01-15T19:10:16.271398+0000","last_peered":"2021-01-15T19:10:16.271398+0000","last_clean":"2021-01-15T19:10:16.271398+0000","last_became_active":"2021-01-15T19:09:51.204375+0000","last_became_peered":"2021-01-15T19:09:51.204375+0000","last_unstale":"2021-01-15T19:10:16.271398+0000","last_undegraded":"2021-01-15T19:10:16.271398+0000","last_fullsized":"2021-01-15T19:10:16.271398+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'46","last_scrub_stamp":"2021-01-15T19:10:03.071240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:03.071240+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3137536,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":37,"num_read_kb":33972,"num_write":15,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"},{"start":"71","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":"162","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.230224+0000","last_change":"2021-01-15T19:10:01.966971+0000","last_active":"2021-01-15T19:10:15.230224+0000","last_peered":"2021-01-15T19:10:15.230224+0000","last_clean":"2021-01-15T19:10:15.230224+0000","last_became_active":"2021-01-15T19:09:48.199625+0000","last_became_peered":"2021-01-15T19:09:48.199625+0000","last_unstale":"2021-01-15T19:10:15.230224+0000","last_undegraded":"2021-01-15T19:10:15.230224+0000","last_fullsized":"2021-01-15T19:10:15.230224+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:01.966888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.966888+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"147'239","reported_seq":"737","reported_epoch":"148","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:16.262714+0000","last_change":"2021-01-15T19:10:16.262761+0000","last_active":"2021-01-15T19:10:16.262714+0000","last_peered":"2021-01-15T19:10:16.262714+0000","last_clean":"2021-01-15T19:10:16.262714+0000","last_became_active":"2021-01-15T19:09:51.211452+0000","last_became_peered":"2021-01-15T19:09:51.211452+0000","last_unstale":"2021-01-15T19:10:16.262714+0000","last_undegraded":"2021-01-15T19:10:16.262714+0000","last_fullsized":"2021-01-15T19:10:16.262714+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'176","last_scrub_stamp":"2021-01-15T19:09:53.888744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.888744+0000","log_size":239,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":25624576,"num_objects":14,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":82,"num_read_kb":111776,"num_write":104,"num_write_kb":32080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":87048192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.d","version":"0'0","reported_seq":"73","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.224729+0000","last_change":"2021-01-15T19:09:25.416216+0000","last_active":"2021-01-15T19:10:14.224729+0000","last_peered":"2021-01-15T19:10:14.224729+0000","last_clean":"2021-01-15T19:10:14.224729+0000","last_became_active":"2021-01-15T19:09:18.191906+0000","last_became_peered":"2021-01-15T19:09:18.191906+0000","last_unstale":"2021-01-15T19:10:14.224729+0000","last_undegraded":"2021-01-15T19:10:14.224729+0000","last_fullsized":"2021-01-15T19:10:14.224729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:25.416133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.416133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"148'167","reported_seq":"528","reported_epoch":"148","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:16.266200+0000","last_change":"2021-01-15T19:10:16.262870+0000","last_active":"2021-01-15T19:10:16.266200+0000","last_peered":"2021-01-15T19:10:16.266200+0000","last_clean":"2021-01-15T19:10:16.266200+0000","last_became_active":"2021-01-15T19:09:15.345068+0000","last_became_peered":"2021-01-15T19:09:15.345068+0000","last_unstale":"2021-01-15T19:10:16.266200+0000","last_undegraded":"2021-01-15T19:10:16.266200+0000","last_fullsized":"2021-01-15T19:10:16.266200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'91","last_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_deep_scrub":"101'91","last_deep_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.016550+0000","log_size":167,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":7569408,"num_objects":8,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":12313,"num_write":85,"num_write_kb":9464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"110","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.230710+0000","last_change":"2021-01-15T19:10:02.931676+0000","last_active":"2021-01-15T19:10:15.230710+0000","last_peered":"2021-01-15T19:10:15.230710+0000","last_clean":"2021-01-15T19:10:15.230710+0000","last_became_active":"2021-01-15T19:09:48.208390+0000","last_became_peered":"2021-01-15T19:09:48.208390+0000","last_unstale":"2021-01-15T19:10:15.230710+0000","last_undegraded":"2021-01-15T19:10:15.230710+0000","last_fullsized":"2021-01-15T19:10:15.230710+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:02.931607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.931607+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"147'95","reported_seq":"471","reported_epoch":"147","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:15.231429+0000","last_change":"2021-01-15T19:10:15.231429+0000","last_active":"2021-01-15T19:10:15.231429+0000","last_peered":"2021-01-15T19:10:15.231429+0000","last_clean":"2021-01-15T19:10:15.231429+0000","last_became_active":"2021-01-15T19:09:21.208779+0000","last_became_peered":"2021-01-15T19:09:21.208779+0000","last_unstale":"2021-01-15T19:10:15.231429+0000","last_undegraded":"2021-01-15T19:10:15.231429+0000","last_fullsized":"2021-01-15T19:10:15.231429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'59","last_scrub_stamp":"2021-01-15T19:09:28.771226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.771226+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":7471104,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":15,"num_read_kb":559,"num_write":25,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.f","version":"0'0","reported_seq":"156","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.231638+0000","last_change":"2021-01-15T19:09:18.202944+0000","last_active":"2021-01-15T19:10:15.231638+0000","last_peered":"2021-01-15T19:10:15.231638+0000","last_clean":"2021-01-15T19:10:15.231638+0000","last_became_active":"2021-01-15T19:09:18.202311+0000","last_became_peered":"2021-01-15T19:09:18.202311+0000","last_unstale":"2021-01-15T19:10:15.231638+0000","last_undegraded":"2021-01-15T19:10:15.231638+0000","last_fullsized":"2021-01-15T19:10:15.231638+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"147'154","reported_seq":"495","reported_epoch":"148","state":"active+clean+snaptrim_wait","last_fresh":"2021-01-15T19:10:16.252521+0000","last_change":"2021-01-15T19:10:16.252592+0000","last_active":"2021-01-15T19:10:16.252521+0000","last_peered":"2021-01-15T19:10:16.252521+0000","last_clean":"2021-01-15T19:10:16.252521+0000","last_became_active":"2021-01-15T19:08:10.704382+0000","last_became_peered":"2021-01-15T19:08:10.704382+0000","last_unstale":"2021-01-15T19:10:16.252521+0000","last_undegraded":"2021-01-15T19:10:16.252521+0000","last_fullsized":"2021-01-15T19:10:16.252521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'93","last_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_deep_scrub":"100'93","last_deep_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.069708+0000","log_size":154,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":4186112,"num_objects":7,"num_object_clones":4,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":30,"num_read_kb":12444,"num_write":74,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]},{"pgid":"2.e","version":"0'0","reported_seq":"169","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216864+0000","last_change":"2021-01-15T19:10:04.241798+0000","last_active":"2021-01-15T19:10:14.216864+0000","last_peered":"2021-01-15T19:10:14.216864+0000","last_clean":"2021-01-15T19:10:14.216864+0000","last_became_active":"2021-01-15T19:09:18.205134+0000","last_became_peered":"2021-01-15T19:09:18.205134+0000","last_unstale":"2021-01-15T19:10:14.216864+0000","last_undegraded":"2021-01-15T19:10:14.216864+0000","last_fullsized":"2021-01-15T19:10:14.216864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:04.241731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:04.241731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"147'106","reported_seq":"432","reported_epoch":"148","state":"active+clean","last_fresh":"2021-01-15T19:10:16.270870+0000","last_change":"2021-01-15T19:10:16.271310+0000","last_active":"2021-01-15T19:10:16.270870+0000","last_peered":"2021-01-15T19:10:16.270870+0000","last_clean":"2021-01-15T19:10:16.270870+0000","last_became_active":"2021-01-15T19:09:51.206324+0000","last_became_peered":"2021-01-15T19:09:51.206324+0000","last_unstale":"2021-01-15T19:10:16.270870+0000","last_undegraded":"2021-01-15T19:10:16.270870+0000","last_fullsized":"2021-01-15T19:10:16.270870+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":106,"ondisk_log_size":106,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8413184,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":21,"num_read_kb":39830,"num_write":40,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"},{"start":"71","length":"1"}]},{"pgid":"2.10","version":"0'0","reported_seq":"115","reported_epoch":"146","state":"active+clean","last_fresh":"2021-01-15T19:10:14.216182+0000","last_change":"2021-01-15T19:10:02.233511+0000","last_active":"2021-01-15T19:10:14.216182+0000","last_peered":"2021-01-15T19:10:14.216182+0000","last_clean":"2021-01-15T19:10:14.216182+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:10:14.216182+0000","last_undegraded":"2021-01-15T19:10:14.216182+0000","last_fullsized":"2021-01-15T19:10:14.216182+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:02.233439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.233439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"120","reported_epoch":"147","state":"active+clean","last_fresh":"2021-01-15T19:10:15.230134+0000","last_change":"2021-01-15T19:09:12.353642+0000","last_active":"2021-01-15T19:10:15.230134+0000","last_peered":"2021-01-15T19:10:15.230134+0000","last_clean":"2021-01-15T19:10:15.230134+0000","last_became_active":"2021-01-15T19:09:12.353214+0000","last_became_peered":"2021-01-15T19:09:12.353214+0000","last_unstale":"2021-01-15T19:10:15.230134+0000","last_undegraded":"2021-01-15T19:10:15.230134+0000","last_fullsized":"2021-01-15T19:10:15.230134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:41.902685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:41.902685+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"147'158","reported_seq":"511","reported_epoch":"147","state":"active+clean+snaptrim","last_fresh":"2021-01-15T19:10:15.257148+0000","last_change":"2021-01-15T19:10:15.246702+0000","last_active":"2021-01-15T19:10:15.257148+0000","last_peered":"2021-01-15T19:10:15.257148+0000","last_clean":"2021-01-15T19:10:15.257148+0000","last_became_active":"2021-01-15T19:09:21.202752+0000","last_became_peered":"2021-01-15T19:09:21.202752+0000","last_unstale":"2021-01-15T19:10:15.257148+0000","last_undegraded":"2021-01-15T19:10:15.257148+0000","last_fullsized":"2021-01-15T19:10:15.257148+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"146'151","last_scrub_stamp":"2021-01-15T19:10:15.176740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:15.176740+0000","log_size":158,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"stat_sum":{"num_bytes":3031040,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":26361,"num_write":14,"num_write_kb":4936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":12124160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":167346176,"num_objects":135,"num_object_clones":90,"num_object_copies":405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":10,"num_read":465,"num_read_kb":439996,"num_write":754,"num_write_kb":176664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":248659968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329211904,"data_stored":573775872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3805,"internal_metadata":0},"log_size":2719,"ondisk_log_size":2719,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3934,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9184,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":22,"acting":22,"num_store_stats":8},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109496,"kb_used_data":73072,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94262344,"statfs":{"total":96636764160,"available":96524640256,"internally_reserved":0,"allocated":74825728,"data_stored":139672723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91400000000000003,"15min":0.92000000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.629},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1519999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.93799999999999994}]},{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97499999999999998,"15min":0.97099999999999997},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.341,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91400000000000003,"15min":0.91000000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":1.1850000000000001}]},{"osd":2,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.98499999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.113},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.91200000000000003,"15min":0.90900000000000003},"min":{"1min":0.73999999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1140000000000001,"5min":1.276,"15min":1.276},"last":0.58799999999999997}]},{"osd":3,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0800000000000001,"15min":1.0900000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1639999999999999,"15min":1.165},"min":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2589999999999999}]},{"osd":4,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.95399999999999996,"15min":0.97099999999999997},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.204,"5min":1.427,"15min":1.427},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.206,"15min":1.23},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.667,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.0660000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.2150000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.466,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.038},{"interface":"front","average":{"1min":1.028,"5min":1.131,"15min":1.1479999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.387,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.876}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.97199999999999998}]}]},{"osd":6,"up_from":0,"seq":566935683077,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72736,"kb_used_data":43544,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":94299104,"statfs":{"total":96636764160,"available":96562282496,"internally_reserved":0,"allocated":44589056,"data_stored":76599450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2729999999999999,"5min":1.452,"15min":1.452},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0169999999999999,"15min":1.0469999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1819999999999999,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.83599999999999997,"15min":0.85499999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.008,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.085,"15min":1.0920000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3480000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.86599999999999999}]},{"osd":3,"last update":"Fri Jan 15 19:09:55 2021","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1479999999999999,"15min":1.1719999999999999},"min":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4910000000000001,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":0.754},{"interface":"front","average":{"1min":1.111,"5min":1.427,"15min":1.4790000000000001},"min":{"1min":0.70199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6519999999999999,"5min":10.584,"15min":10.584},"last":1.115}]},{"osd":4,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.3620000000000001,"15min":1.3879999999999999},"min":{"1min":0.755,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1699999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.288,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.028}]},{"osd":5,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.5469999999999999,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.228,"5min":1.391,"15min":1.4179999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.673,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.0700000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.996}]},{"osd":7,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.488,"15min":1.512},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.7629999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.587,"15min":1.621},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0150000000000001,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.1619999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62480,"kb_used_data":35848,"kb_used_omap":0,"kb_used_meta":26624,"kb_avail":94309360,"statfs":{"total":96636764160,"available":96572784640,"internally_reserved":0,"allocated":36708352,"data_stored":60451987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27262976},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.048,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94199999999999995,"15min":0.95299999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0549999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.181,"15min":1.198},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.506,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.028}]},{"osd":4,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5369999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.282},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.681,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.234},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.3580000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.724,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.387}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.323},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.3480000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4350000000000001,"15min":1.4550000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.6819999999999999,"5min":1.96,"15min":1.96},"last":1.2},{"interface":"front","average":{"1min":1.367,"5min":1.5349999999999999,"15min":1.5629999999999999},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":1.847,"5min":2.024,"15min":2.024},"last":1.4610000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98484,"kb_used_data":63468,"kb_used_omap":0,"kb_used_meta":35008,"kb_avail":94273356,"statfs":{"total":96636764160,"available":96535916544,"internally_reserved":0,"allocated":64991232,"data_stored":115126426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35848192},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.54700000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2629999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.63}]},{"osd":1,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95699999999999996,"15min":0.94899999999999995},"min":{"1min":0.76900000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.82099999999999995,"15min":0.81100000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.089,"15min":1.0900000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.339}]},{"osd":4,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.149,"15min":1.141},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.86299999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.538,"5min":1.859,"15min":1.859},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.29,"15min":1.2829999999999999},"min":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9239999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.222},{"interface":"front","average":{"1min":1.165,"5min":1.204,"15min":1.21},"min":{"1min":0.90100000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.468,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:52 2021","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.365}]},{"osd":7,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3620000000000001,"15min":1.355},"min":{"1min":0.92300000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0510000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.492},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3999999999999999,"15min":1.397},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.2789999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":26404,"kb_used_data":8796,"kb_used_omap":0,"kb_used_meta":17600,"kb_avail":94345436,"statfs":{"total":96636764160,"available":96609726464,"internally_reserved":0,"allocated":9007104,"data_stored":13221011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":18022400},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.194},{"interface":"front","average":{"1min":0.87,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0629999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.2410000000000001}]},{"osd":1,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.052,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.345}]},{"osd":2,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.421,"5min":1.843,"15min":1.843},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.86199999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4330000000000001,"5min":1.911,"15min":1.911},"last":1.706}]},{"osd":4,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.81399999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.429,"5min":2.02,"15min":2.02},"last":1.637},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1659999999999999,"15min":1.1779999999999999},"min":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4870000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3049999999999999,"15min":1.3089999999999999},"min":{"1min":0.98699999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.744},{"interface":"front","average":{"1min":1.29,"5min":1.4490000000000001,"15min":1.4750000000000001},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":1.7410000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.506}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.9179999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.4099999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.389,"15min":1.405},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.6299999999999999,"5min":2.097,"15min":2.097},"last":1.788},{"interface":"front","average":{"1min":1.367,"5min":1.4139999999999999,"15min":1.4219999999999999},"min":{"1min":1.095,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.613}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83980,"kb_used_data":48324,"kb_used_omap":0,"kb_used_meta":35648,"kb_avail":94287860,"statfs":{"total":96636764160,"available":96550768640,"internally_reserved":0,"allocated":49483776,"data_stored":101489811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":36503552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3280000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.181},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.252,"5min":1.409,"15min":1.409},"last":1.4299999999999999}]},{"osd":1,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91400000000000003,"15min":0.90600000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.617,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2050000000000001,"15min":1.1990000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.5629999999999999}]},{"osd":2,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.98299999999999998,"15min":0.99399999999999999},"min":{"1min":0.69699999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.2050000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.097},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.383},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5980000000000001,"5min":2.161,"15min":2.161},"last":1.6499999999999999}]},{"osd":3,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1950000000000001,"15min":1.1990000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.377},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.137,"15min":1.139},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.5880000000000001}]},{"osd":5,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":1.0389999999999999,"15min":1.0700000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3169999999999999,"5min":1.571,"15min":1.571},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.2989999999999999,"15min":1.331},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.0429999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.129},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.254}]},{"osd":7,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.514,"15min":1.556},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.2440000000000002,"5min":2.52,"15min":2.52},"last":1.288},{"interface":"front","average":{"1min":1.202,"5min":1.365,"15min":1.3919999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.7310000000000001}]}]},{"osd":5,"up_from":0,"seq":55834574880,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}]}} 2021-01-15T19:10:20.592 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2021-01-15T19:10:20.592 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 21 2021-01-15T19:10:20.671 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:20.669+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:20.683 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:20.683 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:20.684 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.685 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.426475883999998, 2021-01-15T19:10:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.690 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:20.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:20.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:20.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:20.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:20.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:20.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:20.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:20.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:20.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:20.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:20.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:20.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.720 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.426310771000004, 2021-01-15T19:10:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:20.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:20.724 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.725 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.726 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:20.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:20.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:20.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:20.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:20.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:20.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:20.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:20.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:20.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:20.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:20.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:20.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:20.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:20.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:20.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:20.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:20.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:20.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:20.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:20.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:20.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:20.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:20.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:20.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:20.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:20.758 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.760 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:20.761 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:20.761 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.419030722000002, 2021-01-15T19:10:20.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:20.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.763 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:20.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.764 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:20.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:20.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:20.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:20.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:20.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:20.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:20.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:20.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:20.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:20.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:20.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:20.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:20.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:20.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:20.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:20.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:20.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:20.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:20.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:20.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:20.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:20.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:20.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.779 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.780 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:20.780 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:20.780 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.418904088999994, 2021-01-15T19:10:20.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:20.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:20.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.798 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:20.798 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:20.798 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.336167123999999, 2021-01-15T19:10:20.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:20.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:20.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:20.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:20.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:20.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:20.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:20.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:20.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:20.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:20.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:20.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:20.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:20.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:20.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:20.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:20.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:20.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:20.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:20.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:20.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:20.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:20.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:20.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:20.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.815 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.815 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:20.816 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:20.816 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.336017510000005, 2021-01-15T19:10:20.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:20.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:20.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:20.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.818 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:20.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:20.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:20.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:20.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:20.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:20.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:20.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:20.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:20.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:20.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:20.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:20.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:20.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:20.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:20.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:20.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:20.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:20.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:20.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:20.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:20.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:20.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:20.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:20.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:20.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:20.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:20.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:20.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:20.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.315719827999999, 2021-01-15T19:10:20.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:20.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:20.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:20.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:20.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:20.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:20.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:20.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:20.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:20.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:20.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:20.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:20.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:20.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:20.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:20.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:20.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:20.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:20.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:20.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:20.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:20.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:20.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:20.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:20.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:20.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:20.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:20.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:20.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:20.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.853 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:20.854 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:20.855 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.312663177000005, 2021-01-15T19:10:20.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:20.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:20.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.857 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:20.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:20.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:20.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:20.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:20.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:20.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:20.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:20.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:20.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:20.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:20.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:20.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:20.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:20.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:20.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:20.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:20.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:20.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:20.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:20.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:20.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:20.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:20.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:20.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:20.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:20.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:20.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.874 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:20.875 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:20.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.309649914999994, 2021-01-15T19:10:20.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:20.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.876 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:20.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.878 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:20.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:20.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:20.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:20.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:20.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:20.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:20.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:20.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:20.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:20.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:20.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:20.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:20.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:20.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:20.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:20.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:20.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:20.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:20.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:20.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:20.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:20.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:20.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:20.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:20.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:20.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:20.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.894 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:20.895 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:20.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.306902304000005, 2021-01-15T19:10:20.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:20.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:20.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.898 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:20.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:20.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:20.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:20.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:20.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:20.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:20.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:20.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:20.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:20.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:20.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:20.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:20.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:20.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:20.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:20.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:20.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:20.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:20.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:20.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:20.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:20.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:20.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:20.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:20.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.915 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.916 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:20.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:20.917 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.306468482, 2021-01-15T19:10:20.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:20.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.918 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.918 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:20.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.918 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:20.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:20.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:20.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:20.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:20.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:20.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:20.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:20.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:20.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:20.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:20.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:20.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:20.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:20.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:20.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:20.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:20.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:20.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:20.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:20.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:20.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:20.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:20.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:20.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:20.934 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.935 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.936 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:20.936 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:20.936 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.306344323000005, 2021-01-15T19:10:20.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:20.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:20.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.937 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.938 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:20.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.938 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:20.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:20.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:20.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:20.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:20.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:20.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:20.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:20.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:20.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:20.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:20.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:20.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:20.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:20.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:20.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:20.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:20.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:20.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:20.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:20.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:20.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:20.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:20.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:20.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.961 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:20.961 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:20.962 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 68.201667787999995, 2021-01-15T19:10:20.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:20.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:20.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:20.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:20.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:20.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:20.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.963 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:20.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:20.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:20.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:20.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:20.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:20.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:20.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:20.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:20.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:20.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:20.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:20.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:20.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:20.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:20.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:20.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:20.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:20.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:20.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:20.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:20.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:20.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:20.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:20.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:20.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:20.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:20.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:20.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:20.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:20.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:20.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:20.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:20.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:20.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:20.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:20.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:20.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:20.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:20.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:20.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:20.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.982 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:20.983 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:20.983 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.321446709, 2021-01-15T19:10:20.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:20.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:20.984 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:20.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:20.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:20.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:20.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:20.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:20.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:20.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:20.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:20.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:20.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:20.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:20.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:20.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:20.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:20.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:20.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:20.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:20.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:20.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:20.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:20.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:20.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.993 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:20.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:20.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:20.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:20.995 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.291728551999995, 2021-01-15T19:10:20.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:20.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:20.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:20.996 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:20.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:20.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:20.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:20.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:20.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:20.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:20.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:20.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:20.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:20.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:20.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:20.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:20.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:21.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:21.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:21.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:21.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:21.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:21.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:21.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:21.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:21.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:21.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.006 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.007 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:21.007 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:21.007 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.287642384999998, 2021-01-15T19:10:21.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:21.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.008 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:21.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:21.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:21.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:21.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:21.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:21.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:21.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:21.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:21.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:21.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:21.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:21.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:21.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:21.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:21.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.017 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.017 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.017 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:21.018 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:21.018 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.287506633999996, 2021-01-15T19:10:21.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:21.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.019 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:21.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:21.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:21.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:21.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:21.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:21.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:21.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:21.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:21.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:21.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:21.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:21.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:21.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:21.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:21.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.027 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.028 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:21.028 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:21.028 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.449007650999999, 2021-01-15T19:10:21.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:21.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.029 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.030 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:21.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.030 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:21.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:21.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:21.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:21.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:21.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:21.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:21.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:21.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:21.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:21.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:21.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:21.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:21.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:21.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:21.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:21.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:21.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:21.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:21.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:21.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:21.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:21.041 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.042 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.043 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:21.043 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:21.043 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.474428186999999, 2021-01-15T19:10:21.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:21.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:21.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:21.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.045 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:21.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:21.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:21.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:21.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:21.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:21.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:21.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:21.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:21.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:21.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:21.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:21.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:21.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:21.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:21.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:21.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:21.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:21.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:21.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:21.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:21.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:21.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:21.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:21.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:21.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:21.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:21.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:21.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:21.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.060 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.060 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.060 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:21.061 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:21.061 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 10.474296526, 2021-01-15T19:10:21.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:21.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.062 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:21.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.063 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:21.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:21.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:21.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:21.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:21.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:21.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:21.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:21.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:21.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:21.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:21.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:21.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:21.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:21.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:21.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:21.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:21.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:21.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:21.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:21.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:21.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:21.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:21.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:21.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:21.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:21.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:21.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:21.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:21.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:21.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:21.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:21.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.078 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.079 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.079 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:21.081 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:21.078+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:21.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:10:21.182 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:21.180+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:21.214 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:21.214 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:21.215 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:21.216 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:21.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:10:21.257 INFO:tasks.rados.rados.0.smithi001.stdout:616: done (0 left) 2021-01-15T19:10:21.257 INFO:tasks.rados.rados.0.smithi001.stdout:617: delete oid 27 current snap is 70 2021-01-15T19:10:21.264 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:10:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 1 pg_num to 21 2021-01-15T19:10:21.278 INFO:tasks.rados.rados.0.smithi001.stdout:617: done (0 left) 2021-01-15T19:10:21.279 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 43 current snap is 70 2021-01-15T19:10:21.279 INFO:tasks.rados.rados.0.smithi001.stdout:618: seq_num 219 ranges {4128768=475136,4603904=466944,5070848=335872} 2021-01-15T19:10:21.282 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:21.280+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:21.305 INFO:tasks.rados.rados.0.smithi001.stdout:618: writing smithi00140323-43 from 4128768 to 4603904 tid 1 2021-01-15T19:10:21.308 INFO:tasks.rados.rados.0.smithi001.stdout:618: writing smithi00140323-43 from 4603904 to 5070848 tid 2 2021-01-15T19:10:21.312 INFO:tasks.rados.rados.0.smithi001.stdout:618: writing smithi00140323-43 from 5070848 to 5406720 tid 3 2021-01-15T19:10:21.312 INFO:tasks.rados.rados.0.smithi001.stdout:619: rollback oid 1 current snap is 70 2021-01-15T19:10:21.313 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 1 to 55 2021-01-15T19:10:21.313 INFO:tasks.rados.rados.0.smithi001.stdout:620: read oid 10 snap 57 2021-01-15T19:10:21.313 INFO:tasks.rados.rados.0.smithi001.stdout:620: expect (ObjNum 172 snap 50 seq_num 172) 2021-01-15T19:10:21.314 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 70 2021-01-15T19:10:21.314 INFO:tasks.rados.rados.0.smithi001.stdout:621: seq_num 220 ranges {0=491520,491520=753664,1245184=49152} 2021-01-15T19:10:21.315 INFO:tasks.rados.rados.0.smithi001.stdout:621: writing smithi00140323-39 from 0 to 491520 tid 1 2021-01-15T19:10:21.322 INFO:tasks.rados.rados.0.smithi001.stdout:621: writing smithi00140323-39 from 491520 to 1245184 tid 2 2021-01-15T19:10:21.323 INFO:tasks.rados.rados.0.smithi001.stdout:621: writing smithi00140323-39 from 1245184 to 1294336 tid 3 2021-01-15T19:10:21.323 INFO:tasks.rados.rados.0.smithi001.stdout:618: finishing write tid 1 to smithi00140323-43 2021-01-15T19:10:21.324 INFO:tasks.rados.rados.0.smithi001.stdout:618: finishing write tid 2 to smithi00140323-43 2021-01-15T19:10:21.324 INFO:tasks.rados.rados.0.smithi001.stdout:619: finishing rollback tid 0 to smithi00140323-1 2021-01-15T19:10:21.324 INFO:tasks.rados.rados.0.smithi001.stdout:619: finishing rollback tid 1 to smithi00140323-1 2021-01-15T19:10:21.324 INFO:tasks.rados.rados.0.smithi001.stdout:619: finishing rollback tid 2 to smithi00140323-1 2021-01-15T19:10:21.324 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 1 v 160 (ObjNum 1 snap 32581 seq_num 179745024) dirty exists 2021-01-15T19:10:21.325 INFO:tasks.rados.rados.0.smithi001.stdout:621: finishing write tid 1 to smithi00140323-39 2021-01-15T19:10:21.325 INFO:tasks.rados.rados.0.smithi001.stdout:618: finishing write tid 3 to smithi00140323-43 2021-01-15T19:10:21.325 INFO:tasks.rados.rados.0.smithi001.stdout:618: finishing write tid 4 to smithi00140323-43 2021-01-15T19:10:21.325 INFO:tasks.rados.rados.0.smithi001.stdout:619: done (3 left) 2021-01-15T19:10:21.325 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 17 current snap is 70 2021-01-15T19:10:21.326 INFO:tasks.rados.rados.0.smithi001.stdout:622: seq_num 221 ranges {1548288=507904,2056192=688128,2744320=114688} 2021-01-15T19:10:21.336 INFO:tasks.rados.rados.0.smithi001.stdout:622: writing smithi00140323-17 from 1548288 to 2056192 tid 1 2021-01-15T19:10:21.341 INFO:tasks.rados.rados.0.smithi001.stdout:622: writing smithi00140323-17 from 2056192 to 2744320 tid 2 2021-01-15T19:10:21.342 INFO:tasks.rados.rados.0.smithi001.stdout:622: writing smithi00140323-17 from 2744320 to 2859008 tid 3 2021-01-15T19:10:21.343 INFO:tasks.rados.rados.0.smithi001.stdout:618: finishing write tid 5 to smithi00140323-43 2021-01-15T19:10:21.343 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 282 (ObjNum 219 snap 70 seq_num 219) dirty exists 2021-01-15T19:10:21.343 INFO:tasks.rados.rados.0.smithi001.stdout:618: left oid 43 (ObjNum 219 snap 70 seq_num 219) 2021-01-15T19:10:21.343 INFO:tasks.rados.rados.0.smithi001.stdout:621: finishing write tid 2 to smithi00140323-39 2021-01-15T19:10:21.344 INFO:tasks.rados.rados.0.smithi001.stdout:620: expect (ObjNum 172 snap 50 seq_num 172) 2021-01-15T19:10:21.383 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:21.382+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:21.406 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:21.406 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:21.407 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:21.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:10:21.501 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:21.500+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:21.517 INFO:tasks.rados.rados.0.smithi001.stdout:621: finishing write tid 3 to smithi00140323-39 2021-01-15T19:10:21.518 INFO:tasks.rados.rados.0.smithi001.stdout:621: finishing write tid 4 to smithi00140323-39 2021-01-15T19:10:21.518 INFO:tasks.rados.rados.0.smithi001.stdout:621: finishing write tid 5 to smithi00140323-39 2021-01-15T19:10:21.518 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 73 (ObjNum 220 snap 70 seq_num 220) dirty exists 2021-01-15T19:10:21.519 INFO:tasks.rados.rados.0.smithi001.stdout:621: left oid 39 (ObjNum 220 snap 70 seq_num 220) 2021-01-15T19:10:21.519 INFO:tasks.rados.rados.0.smithi001.stdout:618: done (3 left) 2021-01-15T19:10:21.519 INFO:tasks.rados.rados.0.smithi001.stdout:620: done (2 left) 2021-01-15T19:10:21.519 INFO:tasks.rados.rados.0.smithi001.stdout:621: done (1 left) 2021-01-15T19:10:21.520 INFO:tasks.rados.rados.0.smithi001.stdout:623: copy_from oid 21 from oid 28 current snap is 70 2021-01-15T19:10:21.520 INFO:tasks.rados.rados.0.smithi001.stdout:622: finishing write tid 1 to smithi00140323-17 2021-01-15T19:10:21.520 INFO:tasks.rados.rados.0.smithi001.stdout:622: finishing write tid 2 to smithi00140323-17 2021-01-15T19:10:21.520 INFO:tasks.rados.rados.0.smithi001.stdout:622: finishing write tid 3 to smithi00140323-17 2021-01-15T19:10:21.521 INFO:tasks.rados.rados.0.smithi001.stdout:622: finishing write tid 4 to smithi00140323-17 2021-01-15T19:10:21.521 INFO:tasks.rados.rados.0.smithi001.stdout:622: finishing write tid 5 to smithi00140323-17 2021-01-15T19:10:21.521 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 17 v 137 (ObjNum 221 snap 70 seq_num 221) dirty exists 2021-01-15T19:10:21.521 INFO:tasks.rados.rados.0.smithi001.stdout:622: left oid 17 (ObjNum 221 snap 70 seq_num 221) 2021-01-15T19:10:21.521 INFO:tasks.rados.rados.0.smithi001.stdout:622: done (1 left) 2021-01-15T19:10:21.522 INFO:tasks.rados.rados.0.smithi001.stdout:624: rollback oid 37 current snap is 70 2021-01-15T19:10:21.522 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 37 to 64 2021-01-15T19:10:21.522 INFO:tasks.rados.rados.0.smithi001.stdout:625: setattr oid 32 current snap is 70 2021-01-15T19:10:21.523 INFO:tasks.rados.rados.0.smithi001.stdout:626: snap_create 2021-01-15T19:10:21.530 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 149 (ObjNum 206 snap 64 seq_num 206) dirty exists 2021-01-15T19:10:21.532 INFO:tasks.rados.rados.0.smithi001.stdout:624: finishing rollback tid 0 to smithi00140323-37 2021-01-15T19:10:21.534 INFO:tasks.rados.rados.0.smithi001.stdout:624: finishing rollback tid 1 to smithi00140323-37 2021-01-15T19:10:21.535 INFO:tasks.rados.rados.0.smithi001.stdout:624: finishing rollback tid 2 to smithi00140323-37 2021-01-15T19:10:21.535 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 37 v 141 (ObjNum 176 snap 56 seq_num 176) dirty exists 2021-01-15T19:10:21.555 INFO:tasks.rados.rados.0.smithi001.stdout:623: finishing copy_from to smithi00140323-21 2021-01-15T19:10:21.556 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 21 v 205 (ObjNum 167 snap 47 seq_num 167) dirty exists 2021-01-15T19:10:21.558 INFO:tasks.rados.rados.0.smithi001.stdout:623: finishing copy_from racing read to smithi00140323-21 2021-01-15T19:10:21.585 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:21.584+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:21.607 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:21.607 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:21.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:21.608 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:21.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.609 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:10:21.609 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:21.610 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 151.087410033, 2021-01-15T19:10:21.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:10:21.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:10:21.612 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:10:21.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:10:21.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.614 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:21.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:21.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:10:21.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:10:21.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:10:21.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:10:21.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:10:21.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:10:21.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:10:21.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:10:21.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:10:21.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:10:21.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:10:21.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:21.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:10:21.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:10:21.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:10:21.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:10:21.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:10:21.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:10:21.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.636 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.636 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.637 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:829 3.10s0 3:0c0290e4:::smithi00140323-22:head [read 0~679936,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:21.638 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:21.638 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.421695713000005, 2021-01-15T19:10:21.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001, 2021-01-15T19:10:21.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 829 2021-01-15T19:10:21.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:21.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182569+0000", 2021-01-15T19:10:21.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4360000000000001e-05 2021-01-15T19:10:21.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182623+0000", 2021-01-15T19:10:21.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0169999999999999e-06 2021-01-15T19:10:21.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182630+0000", 2021-01-15T19:10:21.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4198e-05 2021-01-15T19:10:21.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182644+0000", 2021-01-15T19:10:21.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1253999999999999e-05 2021-01-15T19:10:21.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182656+0000", 2021-01-15T19:10:21.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9756999999999999e-05 2021-01-15T19:10:21.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182695+0000", 2021-01-15T19:10:21.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2819999999999996e-06 2021-01-15T19:10:21.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:21.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182704+0000", 2021-01-15T19:10:21.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048051730000000003 2021-01-15T19:10:21.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187509+0000", 2021-01-15T19:10:21.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114002 2021-01-15T19:10:21.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187623+0000", 2021-01-15T19:10:21.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012123558 2021-01-15T19:10:21.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.199746+0000", 2021-01-15T19:10:21.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017177601000000001 2021-01-15T19:10:21.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.662 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.as1 80/21 ECSubWrite(tid=315, reqid=client.4418.0:833, at_version=80'84, trim_to=0'0, roll_forward_to=78'82))", 2021-01-15T19:10:21.662 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:21.662 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.417133746000005, 2021-01-15T19:10:21.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001, 2021-01-15T19:10:21.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.663 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:21.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187131+0000", 2021-01-15T19:10:21.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6477000000000002e-05 2021-01-15T19:10:21.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187187+0000", 2021-01-15T19:10:21.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4749e-05 2021-01-15T19:10:21.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187212+0000", 2021-01-15T19:10:21.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5977e-05 2021-01-15T19:10:21.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187228+0000", 2021-01-15T19:10:21.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0770000000000008e-06 2021-01-15T19:10:21.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187236+0000", 2021-01-15T19:10:21.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014010800000000001 2021-01-15T19:10:21.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187376+0000", 2021-01-15T19:10:21.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8624999999999999e-05 2021-01-15T19:10:21.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187395+0000", 2021-01-15T19:10:21.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013611689 2021-01-15T19:10:21.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201006+0000", 2021-01-15T19:10:21.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7980999999999999e-05 2021-01-15T19:10:21.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.201064+0000", 2021-01-15T19:10:21.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013933683000000001 2021-01-15T19:10:21.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.676 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:971 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:21.676 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:21.676 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.394990352000001, 2021-01-15T19:10:21.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002, 2021-01-15T19:10:21.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 971 2021-01-15T19:10:21.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.679 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:21.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:21.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2442999999999998e-05 2021-01-15T19:10:21.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209337+0000", 2021-01-15T19:10:21.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0950000000000002e-06 2021-01-15T19:10:21.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209346+0000", 2021-01-15T19:10:21.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9199999999999999e-05 2021-01-15T19:10:21.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209365+0000", 2021-01-15T19:10:21.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5750000000000007e-06 2021-01-15T19:10:21.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209374+0000", 2021-01-15T19:10:21.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074262999999999998 2021-01-15T19:10:21.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210117+0000", 2021-01-15T19:10:21.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022372900000000001 2021-01-15T19:10:21.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210341+0000", 2021-01-15T19:10:21.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016740338 2021-01-15T19:10:21.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227081+0000", 2021-01-15T19:10:21.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002 2021-01-15T19:10:21.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.689 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.690 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:972 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:21.690 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:21.690 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.394843596000001, 2021-01-15T19:10:21.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999, 2021-01-15T19:10:21.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.691 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.691 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 972 2021-01-15T19:10:21.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.692 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:21.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:21.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5875000000000003e-05 2021-01-15T19:10:21.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209477+0000", 2021-01-15T19:10:21.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7499999999999992e-06 2021-01-15T19:10:21.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209486+0000", 2021-01-15T19:10:21.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4825e-05 2021-01-15T19:10:21.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209500+0000", 2021-01-15T19:10:21.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:21.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209508+0000", 2021-01-15T19:10:21.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012352109999999999 2021-01-15T19:10:21.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210743+0000", 2021-01-15T19:10:21.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012467700000000001 2021-01-15T19:10:21.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210867+0000", 2021-01-15T19:10:21.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018560825 2021-01-15T19:10:21.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.229428+0000", 2021-01-15T19:10:21.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999 2021-01-15T19:10:21.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.704 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.705 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:973 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:21.705 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:21.705 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 58.394712263999999, 2021-01-15T19:10:21.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001, 2021-01-15T19:10:21.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.706 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 973 2021-01-15T19:10:21.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.707 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:21.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:21.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7021999999999997e-05 2021-01-15T19:10:21.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209609+0000", 2021-01-15T19:10:21.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3140000000000004e-06 2021-01-15T19:10:21.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209618+0000", 2021-01-15T19:10:21.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4259e-05 2021-01-15T19:10:21.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209632+0000", 2021-01-15T19:10:21.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2021-01-15T19:10:21.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209639+0000", 2021-01-15T19:10:21.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039100000000000003 2021-01-15T19:10:21.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213549+0000", 2021-01-15T19:10:21.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124948 2021-01-15T19:10:21.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213674+0000", 2021-01-15T19:10:21.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018190644999999998 2021-01-15T19:10:21.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.231864+0000", 2021-01-15T19:10:21.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001 2021-01-15T19:10:21.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.717 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1093 3.10s0 3:095f0b32:::smithi00140323-47:head [rollback 50] snapc 50=[50,4b,44,41,26] ondisk+write+known_if_redirected e109)", 2021-01-15T19:10:21.718 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:21.718 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.368038130999999, 2021-01-15T19:10:21.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242, 2021-01-15T19:10:21.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1093 2021-01-15T19:10:21.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.720 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:21.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236226+0000", 2021-01-15T19:10:21.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3699999999999998e-05 2021-01-15T19:10:21.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236270+0000", 2021-01-15T19:10:21.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4700000000000008e-06 2021-01-15T19:10:21.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236280+0000", 2021-01-15T19:10:21.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6716999999999999e-05 2021-01-15T19:10:21.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236296+0000", 2021-01-15T19:10:21.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5490000000000006e-06 2021-01-15T19:10:21.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236305+0000", 2021-01-15T19:10:21.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4991999999999998e-05 2021-01-15T19:10:21.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236360+0000", 2021-01-15T19:10:21.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3660000000000003e-06 2021-01-15T19:10:21.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:21.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.236367+0000", 2021-01-15T19:10:21.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.005477549 2021-01-15T19:10:21.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241845+0000", 2021-01-15T19:10:21.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137458 2021-01-15T19:10:21.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.241982+0000", 2021-01-15T19:10:21.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0097224410000000001 2021-01-15T19:10:21.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.251705+0000", 2021-01-15T19:10:21.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015478242 2021-01-15T19:10:21.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.732 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 109/95 ECSubWrite(tid=263, reqid=client.4418.0:1090, at_version=109'60, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:21.733 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:21.733 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.364222493, 2021-01-15T19:10:21.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999, 2021-01-15T19:10:21.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.734 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:21.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:21.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8936e-05 2021-01-15T19:10:21.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240091+0000", 2021-01-15T19:10:21.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1778000000000001e-05 2021-01-15T19:10:21.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240113+0000", 2021-01-15T19:10:21.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6714999999999998e-05 2021-01-15T19:10:21.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240129+0000", 2021-01-15T19:10:21.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:21.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240137+0000", 2021-01-15T19:10:21.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9104999999999998e-05 2021-01-15T19:10:21.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240186+0000", 2021-01-15T19:10:21.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240194+0000", 2021-01-15T19:10:21.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017479262999999998 2021-01-15T19:10:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257674+0000", 2021-01-15T19:10:21.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9951999999999999e-05 2021-01-15T19:10:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257724+0000", 2021-01-15T19:10:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999 2021-01-15T19:10:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 110/95 ECSubWrite(tid=269, reqid=client.4418.0:1098, at_version=110'64, trim_to=0'0, roll_forward_to=109'62))", 2021-01-15T19:10:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 43.410161483000003, 2021-01-15T19:10:21.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999, 2021-01-15T19:10:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194103+0000", 2021-01-15T19:10:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5725e-05 2021-01-15T19:10:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194149+0000", 2021-01-15T19:10:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3603000000000003e-05 2021-01-15T19:10:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194182+0000", 2021-01-15T19:10:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0721e-05 2021-01-15T19:10:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194203+0000", 2021-01-15T19:10:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3929999999999997e-06 2021-01-15T19:10:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194212+0000", 2021-01-15T19:10:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0593000000000006e-05 2021-01-15T19:10:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194293+0000", 2021-01-15T19:10:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3596e-05 2021-01-15T19:10:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.194307+0000", 2021-01-15T19:10:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017420983000000001 2021-01-15T19:10:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211728+0000", 2021-01-15T19:10:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6654999999999999e-05 2021-01-15T19:10:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:38.211784+0000", 2021-01-15T19:10:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681268999999999 2021-01-15T19:10:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.756 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1116 3.4s1 3:21d37027:::smithi00140323-27:head [copy-from ver 54] snapc 57=[57,56,53,50,4b,44,41,26] ondisk+write+known_if_redirected e114)", 2021-01-15T19:10:21.757 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.351722881000001, 2021-01-15T19:10:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999, 2021-01-15T19:10:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:21.758 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1116 2021-01-15T19:10:21.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:21.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:21.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3304000000000002e-05 2021-01-15T19:10:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252605+0000", 2021-01-15T19:10:21.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0723000000000001e-05 2021-01-15T19:10:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252616+0000", 2021-01-15T19:10:21.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0720000000000002e-05 2021-01-15T19:10:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252636+0000", 2021-01-15T19:10:21.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1799e-05 2021-01-15T19:10:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252648+0000", 2021-01-15T19:10:21.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5387000000000003e-05 2021-01-15T19:10:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252714+0000", 2021-01-15T19:10:21.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029657199999999999 2021-01-15T19:10:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253010+0000", 2021-01-15T19:10:21.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030464359 2021-01-15T19:10:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.283475+0000", 2021-01-15T19:10:21.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011930975999999999 2021-01-15T19:10:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295405+0000", 2021-01-15T19:10:21.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2021-01-15T19:10:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:21.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295488+0000", 2021-01-15T19:10:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074145999999999999 2021-01-15T19:10:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296229+0000", 2021-01-15T19:10:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999 2021-01-15T19:10:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1117 3.4s1 3:21d37027:::smithi00140323-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e114)", 2021-01-15T19:10:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 39.351558384999997, 2021-01-15T19:10:21.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003, 2021-01-15T19:10:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1117 2021-01-15T19:10:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3238000000000002e-05 2021-01-15T19:10:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252749+0000", 2021-01-15T19:10:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8820000000000001e-06 2021-01-15T19:10:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252757+0000", 2021-01-15T19:10:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6419e-05 2021-01-15T19:10:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252774+0000", 2021-01-15T19:10:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9879999999999998e-06 2021-01-15T19:10:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252781+0000", 2021-01-15T19:10:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051378399999999999 2021-01-15T19:10:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253294+0000", 2021-01-15T19:10:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8572000000000006e-05 2021-01-15T19:10:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253373+0000", 2021-01-15T19:10:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040965935000000002 2021-01-15T19:10:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294339+0000", 2021-01-15T19:10:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8033000000000001e-05 2021-01-15T19:10:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294437+0000", 2021-01-15T19:10:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018291189999999999 2021-01-15T19:10:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296266+0000", 2021-01-15T19:10:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0880999999999999e-05 2021-01-15T19:10:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296357+0000", 2021-01-15T19:10:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016417 2021-01-15T19:10:21.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296521+0000", 2021-01-15T19:10:21.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003 2021-01-15T19:10:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s0 3.18aae252 (undecoded) ondisk+write+peerstat_old+known_if_redirected e119)", 2021-01-15T19:10:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.324922051999998, 2021-01-15T19:10:21.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263, 2021-01-15T19:10:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:10:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5269000000000003e-05 2021-01-15T19:10:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279388+0000", 2021-01-15T19:10:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4849999999999998e-06 2021-01-15T19:10:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279397+0000", 2021-01-15T19:10:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.63e-05 2021-01-15T19:10:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279414+0000", 2021-01-15T19:10:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279422+0000", 2021-01-15T19:10:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5129e-05 2021-01-15T19:10:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279477+0000", 2021-01-15T19:10:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1559999999999997e-06 2021-01-15T19:10:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279481+0000", 2021-01-15T19:10:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.92165986799999999 2021-01-15T19:10:21.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201141+0000", 2021-01-15T19:10:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9066999999999999e-05 2021-01-15T19:10:21.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201160+0000", 2021-01-15T19:10:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263 2021-01-15T19:10:21.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.803 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4166.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e99)", 2021-01-15T19:10:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.048552829000002, 2021-01-15T19:10:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116, 2021-01-15T19:10:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4166", 2021-01-15T19:10:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4191424063", 2021-01-15T19:10:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4 2021-01-15T19:10:21.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7705999999999999e-05 2021-01-15T19:10:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555739+0000", 2021-01-15T19:10:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8659999999999995e-06 2021-01-15T19:10:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555748+0000", 2021-01-15T19:10:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2021-01-15T19:10:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555761+0000", 2021-01-15T19:10:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9859999999999994e-06 2021-01-15T19:10:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555769+0000", 2021-01-15T19:10:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034585600000000002 2021-01-15T19:10:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556115+0000", 2021-01-15T19:10:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556129+0000", 2021-01-15T19:10:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.65062051300000001 2021-01-15T19:10:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206749+0000", 2021-01-15T19:10:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014163399999999999 2021-01-15T19:10:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206891+0000", 2021-01-15T19:10:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116 2021-01-15T19:10:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.816 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1187 3.12s0 3:4bacf681:::smithi00140323-17:head [setxattr _header (57) in=7b] snapc 64=[64,63,62,60,5f,5c,58,57,56,4b,44,41,26] ondisk+write+known_if_redirected e129)", 2021-01-15T19:10:21.817 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.377879484000001, 2021-01-15T19:10:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999, 2021-01-15T19:10:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:21.818 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1187 2021-01-15T19:10:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.819 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226385+0000", 2021-01-15T19:10:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8656999999999998e-05 2021-01-15T19:10:21.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226444+0000", 2021-01-15T19:10:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:21.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226452+0000", 2021-01-15T19:10:21.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3689000000000001e-05 2021-01-15T19:10:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226466+0000", 2021-01-15T19:10:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8540000000000002e-06 2021-01-15T19:10:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226472+0000", 2021-01-15T19:10:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2799999999999997e-05 2021-01-15T19:10:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226514+0000", 2021-01-15T19:10:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1725999999999999e-05 2021-01-15T19:10:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:21.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226526+0000", 2021-01-15T19:10:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066831169999999997 2021-01-15T19:10:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.233209+0000", 2021-01-15T19:10:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011647090000000001 2021-01-15T19:10:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234374+0000", 2021-01-15T19:10:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00061833799999999996 2021-01-15T19:10:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.234992+0000", 2021-01-15T19:10:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001433157 2021-01-15T19:10:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.236425+0000", 2021-01-15T19:10:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044118559999999996 2021-01-15T19:10:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240837+0000", 2021-01-15T19:10:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152409 2021-01-15T19:10:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.240990+0000", 2021-01-15T19:10:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014604628999999999 2021-01-15T19:10:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1188 3.12s0 3:4bacf681:::smithi00140323-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:21.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.377744224000001, 2021-01-15T19:10:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001, 2021-01-15T19:10:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1188 2021-01-15T19:10:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226520+0000", 2021-01-15T19:10:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0703e-05 2021-01-15T19:10:21.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226561+0000", 2021-01-15T19:10:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5660000000000001e-06 2021-01-15T19:10:21.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226569+0000", 2021-01-15T19:10:21.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3385e-05 2021-01-15T19:10:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226582+0000", 2021-01-15T19:10:21.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226587+0000", 2021-01-15T19:10:21.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.409e-05 2021-01-15T19:10:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226632+0000", 2021-01-15T19:10:21.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1242e-05 2021-01-15T19:10:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.226643+0000", 2021-01-15T19:10:21.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0086436099999999995 2021-01-15T19:10:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235286+0000", 2021-01-15T19:10:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016845599999999999 2021-01-15T19:10:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:21.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.235455+0000", 2021-01-15T19:10:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0060319140000000002 2021-01-15T19:10:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241487+0000", 2021-01-15T19:10:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014580699999999999 2021-01-15T19:10:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.241633+0000", 2021-01-15T19:10:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001527306 2021-01-15T19:10:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.243160+0000", 2021-01-15T19:10:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016639560000000001 2021-01-15T19:10:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1194 3.12s0 3:4e4266df:::smithi00140323-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e129)", 2021-01-15T19:10:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:21.852 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.348001265000001, 2021-01-15T19:10:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019, 2021-01-15T19:10:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.853 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1194 2021-01-15T19:10:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256263+0000", 2021-01-15T19:10:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8844999999999998e-05 2021-01-15T19:10:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256302+0000", 2021-01-15T19:10:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9030000000000004e-06 2021-01-15T19:10:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256310+0000", 2021-01-15T19:10:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4749999999999999e-05 2021-01-15T19:10:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256325+0000", 2021-01-15T19:10:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2370000000000003e-06 2021-01-15T19:10:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.256332+0000", 2021-01-15T19:10:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011409671999999999 2021-01-15T19:10:21.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267742+0000", 2021-01-15T19:10:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017902200000000001 2021-01-15T19:10:21.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.267921+0000", 2021-01-15T19:10:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027501040000000002 2021-01-15T19:10:21.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270671+0000", 2021-01-15T19:10:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020387700000000001 2021-01-15T19:10:21.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.270875+0000", 2021-01-15T19:10:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012356089999999999 2021-01-15T19:10:21.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:57.272110+0000", 2021-01-15T19:10:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015847019 2021-01-15T19:10:21.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1404 3.10s0 3:0e97918f:::smithi00140323-6:head [append 0~704512] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.3798563269999997, 2021-01-15T19:10:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02076683, 2021-01-15T19:10:21.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1404 2021-01-15T19:10:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033569900000000001 2021-01-15T19:10:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224744+0000", 2021-01-15T19:10:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:10:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224753+0000", 2021-01-15T19:10:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6562000000000002e-05 2021-01-15T19:10:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224770+0000", 2021-01-15T19:10:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3300000000000005e-06 2021-01-15T19:10:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224779+0000", 2021-01-15T19:10:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025161600000000002 2021-01-15T19:10:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.225031+0000", 2021-01-15T19:10:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.225039+0000", 2021-01-15T19:10:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013157500000000001 2021-01-15T19:10:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226355+0000", 2021-01-15T19:10:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154186 2021-01-15T19:10:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226509+0000", 2021-01-15T19:10:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012755450000000001 2021-01-15T19:10:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.227784+0000", 2021-01-15T19:10:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016929759999999999 2021-01-15T19:10:21.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244714+0000", 2021-01-15T19:10:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000165193 2021-01-15T19:10:21.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244879+0000", 2021-01-15T19:10:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029558600000000003 2021-01-15T19:10:21.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245175+0000", 2021-01-15T19:10:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02076683 2021-01-15T19:10:21.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.884 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1405 3.10s0 3:0e97918f:::smithi00140323-6:head [append 0~401408] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.3775786500000002, 2021-01-15T19:10:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018945849000000001, 2021-01-15T19:10:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1405 2021-01-15T19:10:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013035499999999999 2021-01-15T19:10:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226816+0000", 2021-01-15T19:10:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.614e-06 2021-01-15T19:10:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226822+0000", 2021-01-15T19:10:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1060000000000006e-06 2021-01-15T19:10:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226831+0000", 2021-01-15T19:10:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7620000000000001e-06 2021-01-15T19:10:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226837+0000", 2021-01-15T19:10:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061165669999999998 2021-01-15T19:10:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.232953+0000", 2021-01-15T19:10:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000602875 2021-01-15T19:10:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.233556+0000", 2021-01-15T19:10:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017456419999999999 2021-01-15T19:10:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.235302+0000", 2021-01-15T19:10:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010073057 2021-01-15T19:10:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245375+0000", 2021-01-15T19:10:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125055 2021-01-15T19:10:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245500+0000", 2021-01-15T19:10:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181600000000001 2021-01-15T19:10:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245632+0000", 2021-01-15T19:10:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018945849000000001 2021-01-15T19:10:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.899 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1406 3.10s0 3:0e97918f:::smithi00140323-6:head [setxattr _header (56) in=7b] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:21.900 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.3774026340000001, 2021-01-15T19:10:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027386166999999999, 2021-01-15T19:10:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.901 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1406 2021-01-15T19:10:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.902 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3294999999999999e-05 2021-01-15T19:10:21.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226895+0000", 2021-01-15T19:10:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6749999999999996e-06 2021-01-15T19:10:21.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226902+0000", 2021-01-15T19:10:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2227e-05 2021-01-15T19:10:21.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226914+0000", 2021-01-15T19:10:21.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7149999999999997e-06 2021-01-15T19:10:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226919+0000", 2021-01-15T19:10:21.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016673698000000001 2021-01-15T19:10:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.243593+0000", 2021-01-15T19:10:21.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134315 2021-01-15T19:10:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.243727+0000", 2021-01-15T19:10:21.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00064922199999999997 2021-01-15T19:10:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244376+0000", 2021-01-15T19:10:21.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023649980000000001 2021-01-15T19:10:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.246741+0000", 2021-01-15T19:10:21.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073842860000000003 2021-01-15T19:10:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:21.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254125+0000", 2021-01-15T19:10:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122736 2021-01-15T19:10:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254248+0000", 2021-01-15T19:10:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027386166999999999 2021-01-15T19:10:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1407 3.10s0 3:0e97918f:::smithi00140323-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e147)", 2021-01-15T19:10:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 6.377305765, 2021-01-15T19:10:21.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028972056999999999, 2021-01-15T19:10:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:21.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1407 2021-01-15T19:10:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:21.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0265e-05 2021-01-15T19:10:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226979+0000", 2021-01-15T19:10:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1500000000000001e-06 2021-01-15T19:10:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226983+0000", 2021-01-15T19:10:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4959999999999999e-06 2021-01-15T19:10:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226991+0000", 2021-01-15T19:10:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.185e-06 2021-01-15T19:10:21.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226994+0000", 2021-01-15T19:10:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01880718 2021-01-15T19:10:21.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245801+0000", 2021-01-15T19:10:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014826300000000001 2021-01-15T19:10:21.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245949+0000", 2021-01-15T19:10:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083453780000000005 2021-01-15T19:10:21.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254295+0000", 2021-01-15T19:10:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7422999999999994e-05 2021-01-15T19:10:21.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254372+0000", 2021-01-15T19:10:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001558717 2021-01-15T19:10:21.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.255931+0000", 2021-01-15T19:10:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028972056999999999 2021-01-15T19:10:21.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:21.930 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:21.930 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:21.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:21.933 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:21.930+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:21.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:22.033 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:22.031+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:22.110 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:22.111 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:22.111 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:22.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:22.124 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:22.134 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:22.132+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:22.145 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:22.166 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:22.185 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:22.207 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:22.228 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:22.235 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:22.233+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:22.250 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:22.257 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:22.258 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:22.268 INFO:tasks.rados.rados.0.smithi001.stdout:623: done (3 left) 2021-01-15T19:10:22.268 INFO:tasks.rados.rados.0.smithi001.stdout:624: done (2 left) 2021-01-15T19:10:22.268 INFO:tasks.rados.rados.0.smithi001.stdout:625: done (1 left) 2021-01-15T19:10:22.268 INFO:tasks.rados.rados.0.smithi001.stdout:626: done (0 left) 2021-01-15T19:10:22.269 INFO:tasks.rados.rados.0.smithi001.stdout:627: snap_remove snap 65 2021-01-15T19:10:22.273 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:10:22.278 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:22.280 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:22.336 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:22.334+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:22.412 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:22.413 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:22.422 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:22.436 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:22.435+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:22.537 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:22.536+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:22.554 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:22.555 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:22.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:22.648 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:22.647+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:22.736 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:22.736 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:22.738 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:22.737+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:22.746 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:22.839 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:22.838+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:22.917 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:22.917 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:22.928 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:22.968 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:22.967+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:23.040 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:23.038+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:23.066 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:23.067 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:23.076 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:23.141 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:23.141+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:23.242 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:23.240+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:23.244 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:23.244 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:23.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:23.269 INFO:tasks.rados.rados.0.smithi001.stdout:627: done (0 left) 2021-01-15T19:10:23.270 INFO:tasks.rados.rados.0.smithi001.stdout:628: delete oid 34 current snap is 71 2021-01-15T19:10:23.296 INFO:tasks.rados.rados.0.smithi001.stdout:628: done (0 left) 2021-01-15T19:10:23.296 INFO:tasks.rados.rados.0.smithi001.stdout:629: delete oid 10 current snap is 71 2021-01-15T19:10:23.304 INFO:tasks.rados.rados.0.smithi001.stdout:629: done (0 left) 2021-01-15T19:10:23.304 INFO:tasks.rados.rados.0.smithi001.stdout:630: read oid 18 snap -1 2021-01-15T19:10:23.304 INFO:tasks.rados.rados.0.smithi001.stdout:630: expect deleted 2021-01-15T19:10:23.304 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 38 current snap is 71 2021-01-15T19:10:23.305 INFO:tasks.rados.rados.0.smithi001.stdout:631: seq_num 224 ranges {5529600=557056,6086656=737280,6823936=65536} 2021-01-15T19:10:23.337 INFO:tasks.rados.rados.0.smithi001.stdout:631: writing smithi00140323-38 from 5529600 to 6086656 tid 1 2021-01-15T19:10:23.341 INFO:tasks.rados.rados.0.smithi001.stdout:631: writing smithi00140323-38 from 6086656 to 6823936 tid 2 2021-01-15T19:10:23.341 INFO:tasks.rados.rados.0.smithi001.stdout:631: writing smithi00140323-38 from 6823936 to 6889472 tid 3 2021-01-15T19:10:23.342 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 19 current snap is 71 2021-01-15T19:10:23.342 INFO:tasks.rados.rados.0.smithi001.stdout:632: seq_num 225 ranges {57344=573440,630784=622592,1253376=180224} 2021-01-15T19:10:23.345 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:23.343+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:23.345 INFO:tasks.rados.rados.0.smithi001.stdout:632: writing smithi00140323-19 from 57344 to 630784 tid 1 2021-01-15T19:10:23.349 INFO:tasks.rados.rados.0.smithi001.stdout:632: writing smithi00140323-19 from 630784 to 1253376 tid 2 2021-01-15T19:10:23.351 INFO:tasks.rados.rados.0.smithi001.stdout:632: writing smithi00140323-19 from 1253376 to 1433600 tid 3 2021-01-15T19:10:23.351 INFO:tasks.rados.rados.0.smithi001.stdout:633: copy_from oid 5 from oid 44 current snap is 71 2021-01-15T19:10:23.351 INFO:tasks.rados.rados.0.smithi001.stdout:631: finishing write tid 1 to smithi00140323-38 2021-01-15T19:10:23.352 INFO:tasks.rados.rados.0.smithi001.stdout:631: finishing write tid 2 to smithi00140323-38 2021-01-15T19:10:23.352 INFO:tasks.rados.rados.0.smithi001.stdout:630: done (3 left) 2021-01-15T19:10:23.352 INFO:tasks.rados.rados.0.smithi001.stdout:634: read oid 37 snap -1 2021-01-15T19:10:23.352 INFO:tasks.rados.rados.0.smithi001.stdout:634: expect (ObjNum 176 snap 56 seq_num 176) 2021-01-15T19:10:23.352 INFO:tasks.rados.rados.0.smithi001.stdout:635: copy_from oid 4 from oid 32 current snap is 71 2021-01-15T19:10:23.353 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 15 current snap is 71 2021-01-15T19:10:23.353 INFO:tasks.rados.rados.0.smithi001.stdout:636: seq_num 226 ranges {0=589824,589824=507904,1097728=294912} 2021-01-15T19:10:23.354 INFO:tasks.rados.rados.0.smithi001.stdout:636: writing smithi00140323-15 from 0 to 589824 tid 1 2021-01-15T19:10:23.358 INFO:tasks.rados.rados.0.smithi001.stdout:636: writing smithi00140323-15 from 589824 to 1097728 tid 2 2021-01-15T19:10:23.360 INFO:tasks.rados.rados.0.smithi001.stdout:636: writing smithi00140323-15 from 1097728 to 1392640 tid 3 2021-01-15T19:10:23.360 INFO:tasks.rados.rados.0.smithi001.stdout:637: delete oid 16 current snap is 71 2021-01-15T19:10:23.360 INFO:tasks.rados.rados.0.smithi001.stdout:631: finishing write tid 3 to smithi00140323-38 2021-01-15T19:10:23.361 INFO:tasks.rados.rados.0.smithi001.stdout:631: finishing write tid 4 to smithi00140323-38 2021-01-15T19:10:23.361 INFO:tasks.rados.rados.0.smithi001.stdout:631: finishing write tid 5 to smithi00140323-38 2021-01-15T19:10:23.361 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 65 (ObjNum 224 snap 71 seq_num 224) dirty exists 2021-01-15T19:10:23.361 INFO:tasks.rados.rados.0.smithi001.stdout:631: left oid 38 (ObjNum 224 snap 71 seq_num 224) 2021-01-15T19:10:23.361 INFO:tasks.rados.rados.0.smithi001.stdout:633: finishing copy_from to smithi00140323-5 2021-01-15T19:10:23.362 INFO:tasks.rados.rados.0.smithi001.stdout:633: got expected ENOENT (src dne) 2021-01-15T19:10:23.362 INFO:tasks.rados.rados.0.smithi001.stdout:633: finishing copy_from racing read to smithi00140323-5 2021-01-15T19:10:23.362 INFO:tasks.rados.rados.0.smithi001.stdout:632: finishing write tid 1 to smithi00140323-19 2021-01-15T19:10:23.363 INFO:tasks.rados.rados.0.smithi001.stdout:632: finishing write tid 2 to smithi00140323-19 2021-01-15T19:10:23.363 INFO:tasks.rados.rados.0.smithi001.stdout:632: finishing write tid 3 to smithi00140323-19 2021-01-15T19:10:23.363 INFO:tasks.rados.rados.0.smithi001.stdout:632: finishing write tid 4 to smithi00140323-19 2021-01-15T19:10:23.365 INFO:tasks.rados.rados.0.smithi001.stdout:631: done (6 left) 2021-01-15T19:10:23.365 INFO:tasks.rados.rados.0.smithi001.stdout:633: done (5 left) 2021-01-15T19:10:23.365 INFO:tasks.rados.rados.0.smithi001.stdout:637: done (4 left) 2021-01-15T19:10:23.366 INFO:tasks.rados.rados.0.smithi001.stdout:638: read oid 2 snap -1 2021-01-15T19:10:23.366 INFO:tasks.rados.rados.0.smithi001.stdout:638: expect (ObjNum 200 snap 61 seq_num 200) 2021-01-15T19:10:23.366 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 18 current snap is 71 2021-01-15T19:10:23.366 INFO:tasks.rados.rados.0.smithi001.stdout:639: seq_num 228 ranges {0=622592,622592=671744,1294336=131072} 2021-01-15T19:10:23.369 INFO:tasks.rados.rados.0.smithi001.stdout:639: writing smithi00140323-18 from 0 to 622592 tid 1 2021-01-15T19:10:23.374 INFO:tasks.rados.rados.0.smithi001.stdout:639: writing smithi00140323-18 from 622592 to 1294336 tid 2 2021-01-15T19:10:23.375 INFO:tasks.rados.rados.0.smithi001.stdout:639: writing smithi00140323-18 from 1294336 to 1425408 tid 3 2021-01-15T19:10:23.376 INFO:tasks.rados.rados.0.smithi001.stdout:640: read oid 48 snap -1 2021-01-15T19:10:23.376 INFO:tasks.rados.rados.0.smithi001.stdout:640: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:10:23.376 INFO:tasks.rados.rados.0.smithi001.stdout:641: read oid 26 snap -1 2021-01-15T19:10:23.376 INFO:tasks.rados.rados.0.smithi001.stdout:641: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:10:23.376 INFO:tasks.rados.rados.0.smithi001.stdout:642: rollback oid 3 current snap is 71 2021-01-15T19:10:23.377 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 3 to 66 2021-01-15T19:10:23.377 INFO:tasks.rados.rados.0.smithi001.stdout:643: read oid 20 snap 57 2021-01-15T19:10:23.377 INFO:tasks.rados.rados.0.smithi001.stdout:643: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:10:23.377 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 33 current snap is 71 2021-01-15T19:10:23.378 INFO:tasks.rados.rados.0.smithi001.stdout:644: seq_num 229 ranges {0=638976,638976=606208,1245184=196608} 2021-01-15T19:10:23.381 INFO:tasks.rados.rados.0.smithi001.stdout:644: writing smithi00140323-33 from 0 to 638976 tid 1 2021-01-15T19:10:23.385 INFO:tasks.rados.rados.0.smithi001.stdout:644: writing smithi00140323-33 from 638976 to 1245184 tid 2 2021-01-15T19:10:23.390 INFO:tasks.rados.rados.0.smithi001.stdout:644: writing smithi00140323-33 from 1245184 to 1441792 tid 3 2021-01-15T19:10:23.397 INFO:tasks.rados.rados.0.smithi001.stdout:632: finishing write tid 5 to smithi00140323-19 2021-01-15T19:10:23.397 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 171 (ObjNum 225 snap 71 seq_num 225) dirty exists 2021-01-15T19:10:23.397 INFO:tasks.rados.rados.0.smithi001.stdout:632: left oid 19 (ObjNum 225 snap 71 seq_num 225) 2021-01-15T19:10:23.398 INFO:tasks.rados.rados.0.smithi001.stdout:634: expect (ObjNum 176 snap 56 seq_num 176) 2021-01-15T19:10:23.444 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:23.444 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:23.446 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:23.445+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:23.453 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:23.547 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:23.546+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:23.578 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:23.578 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:23.587 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:23.648 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:23.646+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:23.711 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:23.712 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:23.722 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:23.763 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:23.761+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:23.774 INFO:tasks.rados.rados.0.smithi001.stdout:632: done (10 left) 2021-01-15T19:10:23.775 INFO:tasks.rados.rados.0.smithi001.stdout:634: done (9 left) 2021-01-15T19:10:23.775 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 42 current snap is 71 2021-01-15T19:10:23.775 INFO:tasks.rados.rados.0.smithi001.stdout:645: seq_num 230 ranges {3325952=655360,3981312=491520,4472832=311296} 2021-01-15T19:10:23.797 INFO:tasks.rados.rados.0.smithi001.stdout:645: writing smithi00140323-42 from 3325952 to 3981312 tid 1 2021-01-15T19:10:23.800 INFO:tasks.rados.rados.0.smithi001.stdout:645: writing smithi00140323-42 from 3981312 to 4472832 tid 2 2021-01-15T19:10:23.803 INFO:tasks.rados.rados.0.smithi001.stdout:645: writing smithi00140323-42 from 4472832 to 4784128 tid 3 2021-01-15T19:10:23.803 INFO:tasks.rados.rados.0.smithi001.stdout:646: read oid 13 snap 67 2021-01-15T19:10:23.804 INFO:tasks.rados.rados.0.smithi001.stdout:646: expect (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:10:23.804 INFO:tasks.rados.rados.0.smithi001.stdout:647: read oid 8 snap 40 2021-01-15T19:10:23.805 INFO:tasks.rados.rados.0.smithi001.stdout:647: expect (ObjNum 131 snap 31 seq_num 131) 2021-01-15T19:10:23.805 INFO:tasks.rados.rados.0.smithi001.stdout:648: read oid 40 snap -1 2021-01-15T19:10:23.806 INFO:tasks.rados.rados.0.smithi001.stdout:648: expect (ObjNum 191 snap 59 seq_num 191) 2021-01-15T19:10:23.806 INFO:tasks.rados.rados.0.smithi001.stdout:649: snap_remove snap 60 2021-01-15T19:10:23.850 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:23.848+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:23.899 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:23.900 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:23.909 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:23.950 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:23.950+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:24.052 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:24.051+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:24.074 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:24.074 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:24.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:24.153 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:24.151+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:24.209 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:24.209 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:24.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:24.253 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:24.253+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:24.276 INFO:tasks.rados.rados.0.smithi001.stdout:649: done (13 left) 2021-01-15T19:10:24.276 INFO:tasks.rados.rados.0.smithi001.stdout:650: setattr oid 41 current snap is 71 2021-01-15T19:10:24.276 INFO:tasks.rados.rados.0.smithi001.stdout:641: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:10:24.289 INFO:tasks.rados.rados.0.smithi001.stdout:642: finishing rollback tid 0 to smithi00140323-3 2021-01-15T19:10:24.289 INFO:tasks.rados.rados.0.smithi001.stdout:639: finishing write tid 1 to smithi00140323-18 2021-01-15T19:10:24.289 INFO:tasks.rados.rados.0.smithi001.stdout:639: finishing write tid 2 to smithi00140323-18 2021-01-15T19:10:24.290 INFO:tasks.rados.rados.0.smithi001.stdout:642: finishing rollback tid 1 to smithi00140323-3 2021-01-15T19:10:24.290 INFO:tasks.rados.rados.0.smithi001.stdout:642: finishing rollback tid 2 to smithi00140323-3 2021-01-15T19:10:24.290 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 88 (ObjNum 170 snap 49 seq_num 170) dirty exists 2021-01-15T19:10:24.290 INFO:tasks.rados.rados.0.smithi001.stdout:639: finishing write tid 3 to smithi00140323-18 2021-01-15T19:10:24.291 INFO:tasks.rados.rados.0.smithi001.stdout:639: finishing write tid 4 to smithi00140323-18 2021-01-15T19:10:24.291 INFO:tasks.rados.rados.0.smithi001.stdout:639: finishing write tid 5 to smithi00140323-18 2021-01-15T19:10:24.291 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 18 v 196 (ObjNum 228 snap 71 seq_num 228) dirty exists 2021-01-15T19:10:24.292 INFO:tasks.rados.rados.0.smithi001.stdout:639: left oid 18 (ObjNum 228 snap 71 seq_num 228) 2021-01-15T19:10:24.292 INFO:tasks.rados.rados.0.smithi001.stdout:640: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:10:24.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:10:24.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:24.354 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:24.352+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:24.394 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:24.394 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:24.403 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:24.454 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:24.453+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:24.491 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:24.510 INFO:tasks.rados.rados.0.smithi001.stdout:638: expect (ObjNum 200 snap 61 seq_num 200) 2021-01-15T19:10:24.512 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1: {} 2021-01-15T19:10:24.533 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:24.552 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:24.555 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:24.553+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:24.568 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:24.568 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:24.573 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:24.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:24.594 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:24.615 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:24.636 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:24.660 INFO:tasks.rados.rados.0.smithi001.stdout:636: finishing write tid 1 to smithi00140323-15 2021-01-15T19:10:24.661 INFO:tasks.rados.rados.0.smithi001.stdout:636: finishing write tid 2 to smithi00140323-15 2021-01-15T19:10:24.661 INFO:tasks.rados.rados.0.smithi001.stdout:636: finishing write tid 3 to smithi00140323-15 2021-01-15T19:10:24.661 INFO:tasks.rados.rados.0.smithi001.stdout:636: finishing write tid 4 to smithi00140323-15 2021-01-15T19:10:24.661 INFO:tasks.rados.rados.0.smithi001.stdout:643: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:10:24.663 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:24.661+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:24.670 INFO:tasks.rados.rados.0.smithi001.stdout:644: finishing write tid 1 to smithi00140323-33 2021-01-15T19:10:24.671 INFO:tasks.rados.rados.0.smithi001.stdout:635: finishing copy_from to smithi00140323-4 2021-01-15T19:10:24.671 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 149 (ObjNum 206 snap 64 seq_num 206) dirty exists 2021-01-15T19:10:24.671 INFO:tasks.rados.rados.0.smithi001.stdout:644: finishing write tid 2 to smithi00140323-33 2021-01-15T19:10:24.671 INFO:tasks.rados.rados.0.smithi001.stdout:644: finishing write tid 3 to smithi00140323-33 2021-01-15T19:10:24.672 INFO:tasks.rados.rados.0.smithi001.stdout:644: finishing write tid 4 to smithi00140323-33 2021-01-15T19:10:24.672 INFO:tasks.rados.rados.0.smithi001.stdout:635: finishing copy_from racing read to smithi00140323-4 2021-01-15T19:10:24.672 INFO:tasks.rados.rados.0.smithi001.stdout:644: finishing write tid 5 to smithi00140323-33 2021-01-15T19:10:24.672 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 265 (ObjNum 229 snap 71 seq_num 229) dirty exists 2021-01-15T19:10:24.672 INFO:tasks.rados.rados.0.smithi001.stdout:644: left oid 33 (ObjNum 229 snap 71 seq_num 229) 2021-01-15T19:10:24.673 INFO:tasks.rados.rados.0.smithi001.stdout:636: finishing write tid 5 to smithi00140323-15 2021-01-15T19:10:24.673 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 15 v 149 (ObjNum 226 snap 71 seq_num 226) dirty exists 2021-01-15T19:10:24.673 INFO:tasks.rados.rados.0.smithi001.stdout:636: left oid 15 (ObjNum 226 snap 71 seq_num 226) 2021-01-15T19:10:24.673 INFO:tasks.rados.rados.0.smithi001.stdout:635: done (13 left) 2021-01-15T19:10:24.673 INFO:tasks.rados.rados.0.smithi001.stdout:636: done (12 left) 2021-01-15T19:10:24.674 INFO:tasks.rados.rados.0.smithi001.stdout:638: done (11 left) 2021-01-15T19:10:24.674 INFO:tasks.rados.rados.0.smithi001.stdout:639: done (10 left) 2021-01-15T19:10:24.674 INFO:tasks.rados.rados.0.smithi001.stdout:640: done (9 left) 2021-01-15T19:10:24.674 INFO:tasks.rados.rados.0.smithi001.stdout:641: done (8 left) 2021-01-15T19:10:24.675 INFO:tasks.rados.rados.0.smithi001.stdout:642: done (7 left) 2021-01-15T19:10:24.675 INFO:tasks.rados.rados.0.smithi001.stdout:643: done (6 left) 2021-01-15T19:10:24.675 INFO:tasks.rados.rados.0.smithi001.stdout:644: done (5 left) 2021-01-15T19:10:24.675 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 3 current snap is 71 2021-01-15T19:10:24.675 INFO:tasks.rados.rados.0.smithi001.stdout:648: expect (ObjNum 191 snap 59 seq_num 191) 2021-01-15T19:10:24.691 INFO:tasks.rados.rados.0.smithi001.stdout:645: finishing write tid 1 to smithi00140323-42 2021-01-15T19:10:24.691 INFO:tasks.rados.rados.0.smithi001.stdout:645: finishing write tid 2 to smithi00140323-42 2021-01-15T19:10:24.691 INFO:tasks.rados.rados.0.smithi001.stdout:645: finishing write tid 3 to smithi00140323-42 2021-01-15T19:10:24.692 INFO:tasks.rados.rados.0.smithi001.stdout:645: finishing write tid 4 to smithi00140323-42 2021-01-15T19:10:24.692 INFO:tasks.rados.rados.0.smithi001.stdout:645: finishing write tid 5 to smithi00140323-42 2021-01-15T19:10:24.692 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 152 (ObjNum 230 snap 71 seq_num 230) dirty exists 2021-01-15T19:10:24.693 INFO:tasks.rados.rados.0.smithi001.stdout:645: left oid 42 (ObjNum 230 snap 71 seq_num 230) 2021-01-15T19:10:24.693 INFO:tasks.rados.rados.0.smithi001.stdout:647: expect (ObjNum 131 snap 31 seq_num 131) 2021-01-15T19:10:24.744 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:24.744 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:24.745 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:24.745 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:24.747 INFO:tasks.rados.rados.0.smithi001.stdout:646: expect (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:10:24.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:24.794 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:24.792+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:24.857 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:24.855+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:24.919 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:24.920 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:24.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:24.920 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:24.920 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:24.929 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:24.958 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:24.956+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:25.059 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:25.057+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:25.061 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:25.061 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:25.062 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:25.062 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:25.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:25.146 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 41 v 116 (ObjNum 179017472 snap 22026 seq_num 607105088) dirty exists 2021-01-15T19:10:25.146 INFO:tasks.rados.rados.0.smithi001.stdout:651: seq_num 231 ranges {802816=671744,1474560=778240,2252800=24576} 2021-01-15T19:10:25.155 INFO:tasks.rados.rados.0.smithi001.stdout:651: writing smithi00140323-3 from 802816 to 1474560 tid 1 2021-01-15T19:10:25.158 INFO:tasks.rados.rados.0.smithi001.stdout:651: writing smithi00140323-3 from 1474560 to 2252800 tid 2 2021-01-15T19:10:25.159 INFO:tasks.rados.rados.0.smithi001.stdout:651: writing smithi00140323-3 from 2252800 to 2277376 tid 3 2021-01-15T19:10:25.159 INFO:tasks.rados.rados.0.smithi001.stdout:645: done (5 left) 2021-01-15T19:10:25.159 INFO:tasks.rados.rados.0.smithi001.stdout:646: done (4 left) 2021-01-15T19:10:25.159 INFO:tasks.rados.rados.0.smithi001.stdout:647: done (3 left) 2021-01-15T19:10:25.160 INFO:tasks.rados.rados.0.smithi001.stdout:648: done (2 left) 2021-01-15T19:10:25.161 INFO:tasks.rados.rados.0.smithi001.stdout:650: done (1 left) 2021-01-15T19:10:25.161 INFO:tasks.rados.rados.0.smithi001.stdout:652: snap_remove snap 52 2021-01-15T19:10:25.161 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:25.158+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:25.246 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:25.248 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:25.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:25.261 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:25.260+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:25.283 INFO:tasks.rados.rados.0.smithi001.stdout:652: done (1 left) 2021-01-15T19:10:25.284 INFO:tasks.rados.rados.0.smithi001.stdout:653: copy_from oid 15 from oid 44 current snap is 71 2021-01-15T19:10:25.284 INFO:tasks.rados.rados.0.smithi001.stdout:651: finishing write tid 1 to smithi00140323-3 2021-01-15T19:10:25.284 INFO:tasks.rados.rados.0.smithi001.stdout:651: finishing write tid 2 to smithi00140323-3 2021-01-15T19:10:25.284 INFO:tasks.rados.rados.0.smithi001.stdout:651: finishing write tid 3 to smithi00140323-3 2021-01-15T19:10:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:651: finishing write tid 4 to smithi00140323-3 2021-01-15T19:10:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:651: finishing write tid 5 to smithi00140323-3 2021-01-15T19:10:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 94 (ObjNum 231 snap 71 seq_num 231) dirty exists 2021-01-15T19:10:25.285 INFO:tasks.rados.rados.0.smithi001.stdout:651: left oid 3 (ObjNum 231 snap 71 seq_num 231) 2021-01-15T19:10:25.286 INFO:tasks.rados.rados.0.smithi001.stdout:651: done (1 left) 2021-01-15T19:10:25.286 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 16 current snap is 71 2021-01-15T19:10:25.286 INFO:tasks.rados.rados.0.smithi001.stdout:654: seq_num 232 ranges {0=688128,688128=663552,1351680=139264} 2021-01-15T19:10:25.286 INFO:tasks.rados.rados.0.smithi001.stdout:654: writing smithi00140323-16 from 0 to 688128 tid 1 2021-01-15T19:10:25.293 INFO:tasks.rados.rados.0.smithi001.stdout:654: writing smithi00140323-16 from 688128 to 1351680 tid 2 2021-01-15T19:10:25.293 INFO:tasks.rados.rados.0.smithi001.stdout:654: writing smithi00140323-16 from 1351680 to 1490944 tid 3 2021-01-15T19:10:25.294 INFO:tasks.rados.rados.0.smithi001.stdout:655: snap_remove snap 67 2021-01-15T19:10:25.294 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:10:25.363 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:25.362+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:25.435 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:25.436 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:25.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:25.437 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:25.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.438 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.179745921000006, 2021-01-15T19:10:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:25.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.440 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:25.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:25.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:25.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:25.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:25.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:25.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:25.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:25.462 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.179580807999997, 2021-01-15T19:10:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:25.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:25.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:25.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:25.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:25.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:25.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:25.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.478 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:25.479 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.172300758999995, 2021-01-15T19:10:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:25.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:25.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:25.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:25.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:25.495 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.172174126000002, 2021-01-15T19:10:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.497 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:25.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:25.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:25.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.512 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 78.089437161000006, 2021-01-15T19:10:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:25.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.528 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 78.089287546999998, 2021-01-15T19:10:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:25.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:25.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:25.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:25.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:25.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:25.545 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.068989865000006, 2021-01-15T19:10:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.546 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:25.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:25.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:25.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:25.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:25.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:25.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:25.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:25.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:25.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.065933213999998, 2021-01-15T19:10:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:25.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:25.583 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.062919952000001, 2021-01-15T19:10:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.584 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:25.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:25.599 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.600 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.060172340999998, 2021-01-15T19:10:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:25.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:25.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:25.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:25.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:25.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.617 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:25.618 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.059738519000007, 2021-01-15T19:10:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.619 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:25.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:25.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:25.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:25.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:25.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:25.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:25.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:25.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:25.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:25.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:25.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.635 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.636 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:25.636 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:25.636 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 73.059614359999998, 2021-01-15T19:10:25.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:25.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.637 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:25.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.638 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:25.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:25.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:25.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:25.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:25.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:25.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:25.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:25.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:25.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:25.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:25.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:25.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:25.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:25.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:25.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:25.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:25.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:25.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:25.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:25.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:25.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:25.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:25.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:25.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:25.651 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.652 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.652 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:25.653 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:25.653 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.954937825000002, 2021-01-15T19:10:25.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:25.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.654 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:25.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.655 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:25.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:25.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:25.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:25.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:25.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:25.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:25.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:25.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:25.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:25.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:25.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:25.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:25.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:25.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:25.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:25.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:25.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:25.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:25.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:25.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:25.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:25.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:25.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:25.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:25.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:25.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:25.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:25.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:25.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:25.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:25.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:25.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:25.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:25.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:25.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:25.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:25.673 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.673 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.674 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:25.674 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:25.675 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.074716746000007, 2021-01-15T19:10:25.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:25.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.675 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:25.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:25.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:25.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:25.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:25.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:25.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:25.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:25.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:25.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:25.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:25.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:25.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:25.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:25.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:25.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:25.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:25.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.685 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:25.686 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:25.686 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.044998589000002, 2021-01-15T19:10:25.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:25.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:25.687 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:25.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:25.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:25.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:25.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:25.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:25.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:25.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:25.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:25.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:25.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:25.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:25.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:25.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:25.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:25.695 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.695 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.695 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.696 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:25.696 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:25.696 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.040912422000005, 2021-01-15T19:10:25.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:25.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:25.697 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:25.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:25.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:25.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:25.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:25.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:25.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:25.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:25.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:25.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:25.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:25.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:25.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:25.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:25.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:25.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.705 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.705 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.706 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:25.706 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:25.706 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 72.040776671000003, 2021-01-15T19:10:25.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:25.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:25.707 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.715 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:25.716 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.202277688000002, 2021-01-15T19:10:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.717 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.718 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:25.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:25.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:25.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:25.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:25.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:25.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:25.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:25.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:25.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:25.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:25.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.227698223999999, 2021-01-15T19:10:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:25.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:25.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:25.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:25.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:25.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:25.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 15.227566563, 2021-01-15T19:10:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:25.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:25.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:25.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:25.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:25.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:25.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:25.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:25.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:25.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:25.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:25.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:25.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:25.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:25.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:25.765 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:25.766 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:25.767 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:25.766+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:25.768 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2021-01-15T19:10:25.868 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:25.867+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:25.944 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:25.944 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:25.945 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:25.945 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:25.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2021-01-15T19:10:25.997 INFO:tasks.ceph.osd.1.smithi001.stderr:2021-01-15T19:10:25.996+0000 7fed5d20d700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 34828) UID: 0 2021-01-15T19:10:26.070 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:26.068+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:26.085 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:26.086 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:26.086 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:26.086 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:26.087 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:26.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2021-01-15T19:10:26.180 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:26.178+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:26.269 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:26.269 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:26.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:26.270 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:26.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4245.0:1 1.0 1:bdc5cd3e:::INTEL_SSDPED1D480GA_PHMB7513000D480DGN:head [omap-get-keys] snapc 0=[] ondisk+read+known_if_redirected e13)", 2021-01-15T19:10:26.270 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:26.271 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 155.74993836199999, 2021-01-15T19:10:26.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999, 2021-01-15T19:10:26.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:26.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4245", 2021-01-15T19:10:26.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/469065039", 2021-01-15T19:10:26.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1 2021-01-15T19:10:26.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.272 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:26.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516854+0000", 2021-01-15T19:10:26.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0182000000000002e-05 2021-01-15T19:10:26.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516905+0000", 2021-01-15T19:10:26.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8910000000000001e-06 2021-01-15T19:10:26.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516914+0000", 2021-01-15T19:10:26.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5783000000000001e-05 2021-01-15T19:10:26.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516929+0000", 2021-01-15T19:10:26.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4879999999999996e-06 2021-01-15T19:10:26.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.516939+0000", 2021-01-15T19:10:26.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0037384060000000001 2021-01-15T19:10:26.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520677+0000", 2021-01-15T19:10:26.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8196e-05 2021-01-15T19:10:26.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:26.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:50.520695+0000", 2021-01-15T19:10:26.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.409205711 2021-01-15T19:10:26.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.929901+0000", 2021-01-15T19:10:26.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000194157 2021-01-15T19:10:26.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:07:51.930095+0000", 2021-01-15T19:10:26.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4132408139999999 2021-01-15T19:10:26.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.286 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:971 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:26.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:26.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.057518680999998, 2021-01-15T19:10:26.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002, 2021-01-15T19:10:26.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 971 2021-01-15T19:10:26.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:26.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209274+0000", 2021-01-15T19:10:26.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2442999999999998e-05 2021-01-15T19:10:26.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209337+0000", 2021-01-15T19:10:26.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0950000000000002e-06 2021-01-15T19:10:26.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209346+0000", 2021-01-15T19:10:26.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9199999999999999e-05 2021-01-15T19:10:26.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209365+0000", 2021-01-15T19:10:26.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5750000000000007e-06 2021-01-15T19:10:26.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209374+0000", 2021-01-15T19:10:26.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074262999999999998 2021-01-15T19:10:26.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210117+0000", 2021-01-15T19:10:26.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022372900000000001 2021-01-15T19:10:26.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210341+0000", 2021-01-15T19:10:26.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016740338 2021-01-15T19:10:26.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.227081+0000", 2021-01-15T19:10:26.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017807010000000002 2021-01-15T19:10:26.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.300 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:972 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:26.300 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:26.300 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.057371924999998, 2021-01-15T19:10:26.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999, 2021-01-15T19:10:26.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.302 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 972 2021-01-15T19:10:26.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.302 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:26.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209421+0000", 2021-01-15T19:10:26.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5875000000000003e-05 2021-01-15T19:10:26.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209477+0000", 2021-01-15T19:10:26.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7499999999999992e-06 2021-01-15T19:10:26.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209486+0000", 2021-01-15T19:10:26.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4825e-05 2021-01-15T19:10:26.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209500+0000", 2021-01-15T19:10:26.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1899999999999998e-06 2021-01-15T19:10:26.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209508+0000", 2021-01-15T19:10:26.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012352109999999999 2021-01-15T19:10:26.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210743+0000", 2021-01-15T19:10:26.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012467700000000001 2021-01-15T19:10:26.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.210867+0000", 2021-01-15T19:10:26.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018560825 2021-01-15T19:10:26.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.229428+0000", 2021-01-15T19:10:26.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.020007352999999999 2021-01-15T19:10:26.313 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.313 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.313 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.314 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:973 3.10s0 3:095f0b32:::smithi00140323-47:head [read 0~3350528,checksum] snapc 0=[] ondisk+read+known_if_redirected e95)", 2021-01-15T19:10:26.314 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:26.314 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 63.057240593000003, 2021-01-15T19:10:26.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001, 2021-01-15T19:10:26.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.315 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 973 2021-01-15T19:10:26.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.316 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:26.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209552+0000", 2021-01-15T19:10:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7021999999999997e-05 2021-01-15T19:10:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209609+0000", 2021-01-15T19:10:26.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3140000000000004e-06 2021-01-15T19:10:26.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209618+0000", 2021-01-15T19:10:26.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4259e-05 2021-01-15T19:10:26.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209632+0000", 2021-01-15T19:10:26.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9e-06 2021-01-15T19:10:26.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.209639+0000", 2021-01-15T19:10:26.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0039100000000000003 2021-01-15T19:10:26.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213549+0000", 2021-01-15T19:10:26.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124948 2021-01-15T19:10:26.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.213674+0000", 2021-01-15T19:10:26.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018190644999999998 2021-01-15T19:10:26.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:23.231864+0000", 2021-01-15T19:10:26.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022312088000000001 2021-01-15T19:10:26.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.327 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWrite(3.11s2 109/95 ECSubWrite(tid=263, reqid=client.4418.0:1090, at_version=109'60, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:26.327 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:26.327 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 49.026750821999997, 2021-01-15T19:10:26.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999, 2021-01-15T19:10:26.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:26.328 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240042+0000", 2021-01-15T19:10:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8936e-05 2021-01-15T19:10:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240091+0000", 2021-01-15T19:10:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1778000000000001e-05 2021-01-15T19:10:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240113+0000", 2021-01-15T19:10:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6714999999999998e-05 2021-01-15T19:10:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240129+0000", 2021-01-15T19:10:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:26.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240137+0000", 2021-01-15T19:10:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9104999999999998e-05 2021-01-15T19:10:26.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240186+0000", 2021-01-15T19:10:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:26.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.240194+0000", 2021-01-15T19:10:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017479262999999998 2021-01-15T19:10:26.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257674+0000", 2021-01-15T19:10:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9951999999999999e-05 2021-01-15T19:10:26.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:37.257724+0000", 2021-01-15T19:10:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017681558999999999 2021-01-15T19:10:26.338 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.339 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1116 3.4s1 3:21d37027:::smithi00140323-27:head [copy-from ver 54] snapc 57=[57,56,53,50,4b,44,41,26] ondisk+write+known_if_redirected e114)", 2021-01-15T19:10:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.014251209999998, 2021-01-15T19:10:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999, 2021-01-15T19:10:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1116 2021-01-15T19:10:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252542+0000", 2021-01-15T19:10:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3304000000000002e-05 2021-01-15T19:10:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252605+0000", 2021-01-15T19:10:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0723000000000001e-05 2021-01-15T19:10:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252616+0000", 2021-01-15T19:10:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0720000000000002e-05 2021-01-15T19:10:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252636+0000", 2021-01-15T19:10:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1799e-05 2021-01-15T19:10:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252648+0000", 2021-01-15T19:10:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5387000000000003e-05 2021-01-15T19:10:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252714+0000", 2021-01-15T19:10:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029657199999999999 2021-01-15T19:10:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253010+0000", 2021-01-15T19:10:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030464359 2021-01-15T19:10:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.283475+0000", 2021-01-15T19:10:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011930975999999999 2021-01-15T19:10:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295405+0000", 2021-01-15T19:10:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2249000000000005e-05 2021-01-15T19:10:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.295488+0000", 2021-01-15T19:10:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074145999999999999 2021-01-15T19:10:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296229+0000", 2021-01-15T19:10:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043687548999999999 2021-01-15T19:10:26.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.355 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1117 3.4s1 3:21d37027:::smithi00140323-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e114)", 2021-01-15T19:10:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 44.014086714000001, 2021-01-15T19:10:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003, 2021-01-15T19:10:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1117 2021-01-15T19:10:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:26.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252706+0000", 2021-01-15T19:10:26.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3238000000000002e-05 2021-01-15T19:10:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252749+0000", 2021-01-15T19:10:26.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8820000000000001e-06 2021-01-15T19:10:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252757+0000", 2021-01-15T19:10:26.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6419e-05 2021-01-15T19:10:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252774+0000", 2021-01-15T19:10:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9879999999999998e-06 2021-01-15T19:10:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.252781+0000", 2021-01-15T19:10:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00051378399999999999 2021-01-15T19:10:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253294+0000", 2021-01-15T19:10:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8572000000000006e-05 2021-01-15T19:10:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:26.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.253373+0000", 2021-01-15T19:10:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040965935000000002 2021-01-15T19:10:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294339+0000", 2021-01-15T19:10:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8033000000000001e-05 2021-01-15T19:10:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:26.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.294437+0000", 2021-01-15T19:10:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018291189999999999 2021-01-15T19:10:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296266+0000", 2021-01-15T19:10:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0880999999999999e-05 2021-01-15T19:10:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296357+0000", 2021-01-15T19:10:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016417 2021-01-15T19:10:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:42.296521+0000", 2021-01-15T19:10:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043815021000000003 2021-01-15T19:10:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1146 3.12s0 3.18aae252 (undecoded) ondisk+write+peerstat_old+known_if_redirected e119)", 2021-01-15T19:10:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.987450381000002, 2021-01-15T19:10:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263, 2021-01-15T19:10:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.373 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1146 2021-01-15T19:10:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279342+0000", 2021-01-15T19:10:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5269000000000003e-05 2021-01-15T19:10:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279388+0000", 2021-01-15T19:10:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4849999999999998e-06 2021-01-15T19:10:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279397+0000", 2021-01-15T19:10:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.63e-05 2021-01-15T19:10:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279414+0000", 2021-01-15T19:10:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279422+0000", 2021-01-15T19:10:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5129e-05 2021-01-15T19:10:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279477+0000", 2021-01-15T19:10:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1559999999999997e-06 2021-01-15T19:10:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.279481+0000", 2021-01-15T19:10:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.92165986799999999 2021-01-15T19:10:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201141+0000", 2021-01-15T19:10:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9066999999999999e-05 2021-01-15T19:10:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.201160+0000", 2021-01-15T19:10:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.921817263 2021-01-15T19:10:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4166.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e99)", 2021-01-15T19:10:26.385 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.711081157999999, 2021-01-15T19:10:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116, 2021-01-15T19:10:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4166", 2021-01-15T19:10:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/4191424063", 2021-01-15T19:10:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 4 2021-01-15T19:10:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.387 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555712+0000", 2021-01-15T19:10:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.7705999999999999e-05 2021-01-15T19:10:26.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555739+0000", 2021-01-15T19:10:26.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8659999999999995e-06 2021-01-15T19:10:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555748+0000", 2021-01-15T19:10:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2680000000000001e-05 2021-01-15T19:10:26.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555761+0000", 2021-01-15T19:10:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9859999999999994e-06 2021-01-15T19:10:26.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.555769+0000", 2021-01-15T19:10:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034585600000000002 2021-01-15T19:10:26.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556115+0000", 2021-01-15T19:10:26.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:47.556129+0000", 2021-01-15T19:10:26.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.65062051300000001 2021-01-15T19:10:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206749+0000", 2021-01-15T19:10:26.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014163399999999999 2021-01-15T19:10:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:48.206891+0000", 2021-01-15T19:10:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.651179116 2021-01-15T19:10:26.397 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1404 3.10s0 3:0e97918f:::smithi00140323-6:head [append 0~704512] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.042384655999999, 2021-01-15T19:10:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02076683, 2021-01-15T19:10:26.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1404 2021-01-15T19:10:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224408+0000", 2021-01-15T19:10:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00033569900000000001 2021-01-15T19:10:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224744+0000", 2021-01-15T19:10:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:10:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224753+0000", 2021-01-15T19:10:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6562000000000002e-05 2021-01-15T19:10:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224770+0000", 2021-01-15T19:10:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3300000000000005e-06 2021-01-15T19:10:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.224779+0000", 2021-01-15T19:10:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025161600000000002 2021-01-15T19:10:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.225031+0000", 2021-01-15T19:10:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.225039+0000", 2021-01-15T19:10:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013157500000000001 2021-01-15T19:10:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226355+0000", 2021-01-15T19:10:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154186 2021-01-15T19:10:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226509+0000", 2021-01-15T19:10:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012755450000000001 2021-01-15T19:10:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.227784+0000", 2021-01-15T19:10:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016929759999999999 2021-01-15T19:10:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244714+0000", 2021-01-15T19:10:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000165193 2021-01-15T19:10:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244879+0000", 2021-01-15T19:10:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00029558600000000003 2021-01-15T19:10:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245175+0000", 2021-01-15T19:10:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02076683 2021-01-15T19:10:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.415 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1405 3.10s0 3:0e97918f:::smithi00140323-6:head [append 0~401408] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:26.416 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.040106979000001, 2021-01-15T19:10:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018945849000000001, 2021-01-15T19:10:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.417 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1405 2021-01-15T19:10:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.418 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226686+0000", 2021-01-15T19:10:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013035499999999999 2021-01-15T19:10:26.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226816+0000", 2021-01-15T19:10:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.614e-06 2021-01-15T19:10:26.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226822+0000", 2021-01-15T19:10:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1060000000000006e-06 2021-01-15T19:10:26.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226831+0000", 2021-01-15T19:10:26.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7620000000000001e-06 2021-01-15T19:10:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226837+0000", 2021-01-15T19:10:26.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061165669999999998 2021-01-15T19:10:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.232953+0000", 2021-01-15T19:10:26.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000602875 2021-01-15T19:10:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.233556+0000", 2021-01-15T19:10:26.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017456419999999999 2021-01-15T19:10:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.235302+0000", 2021-01-15T19:10:26.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010073057 2021-01-15T19:10:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245375+0000", 2021-01-15T19:10:26.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000125055 2021-01-15T19:10:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245500+0000", 2021-01-15T19:10:26.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013181600000000001 2021-01-15T19:10:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245632+0000", 2021-01-15T19:10:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018945849000000001 2021-01-15T19:10:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1406 3.10s0 3:0e97918f:::smithi00140323-6:head [setxattr _header (56) in=7b] snapc 73=[73,72,71,70,6f,6e,6b,67,66,64,63,60,5f,56,4b] ondisk+write+known_if_redirected e147)", 2021-01-15T19:10:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.039930963, 2021-01-15T19:10:26.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027386166999999999, 2021-01-15T19:10:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.433 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1406 2021-01-15T19:10:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226862+0000", 2021-01-15T19:10:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.3294999999999999e-05 2021-01-15T19:10:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226895+0000", 2021-01-15T19:10:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6749999999999996e-06 2021-01-15T19:10:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226902+0000", 2021-01-15T19:10:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2227e-05 2021-01-15T19:10:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226914+0000", 2021-01-15T19:10:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7149999999999997e-06 2021-01-15T19:10:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226919+0000", 2021-01-15T19:10:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016673698000000001 2021-01-15T19:10:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.243593+0000", 2021-01-15T19:10:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134315 2021-01-15T19:10:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.243727+0000", 2021-01-15T19:10:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00064922199999999997 2021-01-15T19:10:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.244376+0000", 2021-01-15T19:10:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0023649980000000001 2021-01-15T19:10:26.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.246741+0000", 2021-01-15T19:10:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0073842860000000003 2021-01-15T19:10:26.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254125+0000", 2021-01-15T19:10:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122736 2021-01-15T19:10:26.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254248+0000", 2021-01-15T19:10:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027386166999999999 2021-01-15T19:10:26.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.447 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1407 3.10s0 3:0e97918f:::smithi00140323-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e147)", 2021-01-15T19:10:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 11.039834094, 2021-01-15T19:10:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028972056999999999, 2021-01-15T19:10:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1407 2021-01-15T19:10:26.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:26.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226959+0000", 2021-01-15T19:10:26.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0265e-05 2021-01-15T19:10:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226979+0000", 2021-01-15T19:10:26.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1500000000000001e-06 2021-01-15T19:10:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226983+0000", 2021-01-15T19:10:26.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4959999999999999e-06 2021-01-15T19:10:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226991+0000", 2021-01-15T19:10:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.185e-06 2021-01-15T19:10:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.226994+0000", 2021-01-15T19:10:26.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01880718 2021-01-15T19:10:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245801+0000", 2021-01-15T19:10:26.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014826300000000001 2021-01-15T19:10:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.245949+0000", 2021-01-15T19:10:26.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0083453780000000005 2021-01-15T19:10:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254295+0000", 2021-01-15T19:10:26.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7422999999999994e-05 2021-01-15T19:10:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.254372+0000", 2021-01-15T19:10:26.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001558717 2021-01-15T19:10:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:15.255931+0000", 2021-01-15T19:10:26.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028972056999999999 2021-01-15T19:10:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1571 3.12s0 3:4a475518:::smithi00140323-15:head [copy-from ver 0] snapc 7d=[7d,7b,7a,79,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e157)", 2021-01-15T19:10:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.278497+0000", 2021-01-15T19:10:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98829621700000003, 2021-01-15T19:10:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02140659, 2021-01-15T19:10:26.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.463 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1571 2021-01-15T19:10:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278497+0000", 2021-01-15T19:10:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278497+0000", 2021-01-15T19:10:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6106000000000002e-05 2021-01-15T19:10:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278543+0000", 2021-01-15T19:10:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3249999999999997e-06 2021-01-15T19:10:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278552+0000", 2021-01-15T19:10:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5675000000000002e-05 2021-01-15T19:10:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278568+0000", 2021-01-15T19:10:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3410000000000003e-06 2021-01-15T19:10:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278576+0000", 2021-01-15T19:10:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2539999999999994e-05 2021-01-15T19:10:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278639+0000", 2021-01-15T19:10:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0508e-05 2021-01-15T19:10:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278649+0000", 2021-01-15T19:10:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0066321660000000001 2021-01-15T19:10:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.285281+0000", 2021-01-15T19:10:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625599999999999 2021-01-15T19:10:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.285408+0000", 2021-01-15T19:10:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014495673000000001 2021-01-15T19:10:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.299903+0000", 2021-01-15T19:10:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.02140659 2021-01-15T19:10:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1572 3.12s0 3:4a475518:::smithi00140323-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e157)", 2021-01-15T19:10:26.476 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.278627+0000", 2021-01-15T19:10:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98816635399999997, 2021-01-15T19:10:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021674133000000002, 2021-01-15T19:10:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1572 2021-01-15T19:10:26.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278627+0000", 2021-01-15T19:10:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278627+0000", 2021-01-15T19:10:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8111000000000002e-05 2021-01-15T19:10:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278665+0000", 2021-01-15T19:10:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0509999999999997e-06 2021-01-15T19:10:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278673+0000", 2021-01-15T19:10:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3193999999999999e-05 2021-01-15T19:10:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278686+0000", 2021-01-15T19:10:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1460000000000004e-06 2021-01-15T19:10:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.278691+0000", 2021-01-15T19:10:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00086608200000000001 2021-01-15T19:10:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.279557+0000", 2021-01-15T19:10:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1392e-05 2021-01-15T19:10:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.279568+0000", 2021-01-15T19:10:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006042576 2021-01-15T19:10:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.285611+0000", 2021-01-15T19:10:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2620999999999996e-05 2021-01-15T19:10:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.285674+0000", 2021-01-15T19:10:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014376813 2021-01-15T19:10:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.300050+0000", 2021-01-15T19:10:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112613 2021-01-15T19:10:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.300163+0000", 2021-01-15T19:10:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137534 2021-01-15T19:10:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.300301+0000", 2021-01-15T19:10:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021674133000000002 2021-01-15T19:10:26.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.493 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.493 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1573 3.10s0 3:0c50693c:::smithi00140323-16:head [append 0~688128] snapc 7d=[7d,7b,7a,79,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e157)", 2021-01-15T19:10:26.494 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.286100+0000", 2021-01-15T19:10:26.494 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.98069257200000004, 2021-01-15T19:10:26.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025800113, 2021-01-15T19:10:26.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.495 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1573 2021-01-15T19:10:26.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.496 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.496 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.286100+0000", 2021-01-15T19:10:26.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.286100+0000", 2021-01-15T19:10:26.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00040327199999999998 2021-01-15T19:10:26.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.286504+0000", 2021-01-15T19:10:26.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4509999999999998e-06 2021-01-15T19:10:26.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.286513+0000", 2021-01-15T19:10:26.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6288000000000001e-05 2021-01-15T19:10:26.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.286529+0000", 2021-01-15T19:10:26.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4989999999999997e-06 2021-01-15T19:10:26.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.286537+0000", 2021-01-15T19:10:26.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025871449999999999 2021-01-15T19:10:26.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.289124+0000", 2021-01-15T19:10:26.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010337599999999999 2021-01-15T19:10:26.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.289227+0000", 2021-01-15T19:10:26.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00091200000000000005 2021-01-15T19:10:26.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290139+0000", 2021-01-15T19:10:26.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012310341000000001 2021-01-15T19:10:26.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.302450+0000", 2021-01-15T19:10:26.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093637010000000003 2021-01-15T19:10:26.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.311813+0000", 2021-01-15T19:10:26.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7039999999999993e-05 2021-01-15T19:10:26.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.311900+0000", 2021-01-15T19:10:26.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025800113 2021-01-15T19:10:26.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.508 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.508 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.509 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1574 3.10s0 3:0c50693c:::smithi00140323-16:head [append 0~663552] snapc 7d=[7d,7b,7a,79,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e157)", 2021-01-15T19:10:26.509 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.288110+0000", 2021-01-15T19:10:26.509 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97868272599999995, 2021-01-15T19:10:26.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024584137999999998, 2021-01-15T19:10:26.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.510 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1574 2021-01-15T19:10:26.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.511 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.288110+0000", 2021-01-15T19:10:26.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.288110+0000", 2021-01-15T19:10:26.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00039076200000000001 2021-01-15T19:10:26.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.288501+0000", 2021-01-15T19:10:26.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3899999999999993e-06 2021-01-15T19:10:26.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.288509+0000", 2021-01-15T19:10:26.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4633e-05 2021-01-15T19:10:26.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.288524+0000", 2021-01-15T19:10:26.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1779999999999997e-06 2021-01-15T19:10:26.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.288531+0000", 2021-01-15T19:10:26.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0038613929999999999 2021-01-15T19:10:26.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.292392+0000", 2021-01-15T19:10:26.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012851 2021-01-15T19:10:26.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.292521+0000", 2021-01-15T19:10:26.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0026511099999999999 2021-01-15T19:10:26.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.295172+0000", 2021-01-15T19:10:26.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072795350000000002 2021-01-15T19:10:26.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.302452+0000", 2021-01-15T19:10:26.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010136910000000001 2021-01-15T19:10:26.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.312589+0000", 2021-01-15T19:10:26.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105717 2021-01-15T19:10:26.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.312694+0000", 2021-01-15T19:10:26.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024584137999999998 2021-01-15T19:10:26.523 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.524 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.524 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1575 3.10s0 3:0c50693c:::smithi00140323-16:head [append 0~139264] snapc 7d=[7d,7b,7a,79,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e157)", 2021-01-15T19:10:26.525 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.290401+0000", 2021-01-15T19:10:26.525 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97639192500000005, 2021-01-15T19:10:26.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023556647999999999, 2021-01-15T19:10:26.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.526 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.526 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.526 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1575 2021-01-15T19:10:26.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290401+0000", 2021-01-15T19:10:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290401+0000", 2021-01-15T19:10:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013879100000000001 2021-01-15T19:10:26.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290540+0000", 2021-01-15T19:10:26.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2199999999999992e-06 2021-01-15T19:10:26.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290548+0000", 2021-01-15T19:10:26.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4918e-05 2021-01-15T19:10:26.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290563+0000", 2021-01-15T19:10:26.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5830000000000003e-06 2021-01-15T19:10:26.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290569+0000", 2021-01-15T19:10:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0077336089999999998 2021-01-15T19:10:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.298303+0000", 2021-01-15T19:10:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016076600000000001 2021-01-15T19:10:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.298464+0000", 2021-01-15T19:10:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00068955700000000004 2021-01-15T19:10:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.299153+0000", 2021-01-15T19:10:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032992479999999999 2021-01-15T19:10:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.302453+0000", 2021-01-15T19:10:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011407607 2021-01-15T19:10:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.313860+0000", 2021-01-15T19:10:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7349000000000006e-05 2021-01-15T19:10:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.313958+0000", 2021-01-15T19:10:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023556647999999999 2021-01-15T19:10:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1576 3.10s0 3:0c50693c:::smithi00140323-16:head [setxattr _header (57) in=7b] snapc 7d=[7d,7b,7a,79,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e157)", 2021-01-15T19:10:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.290615+0000", 2021-01-15T19:10:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97617823000000004, 2021-01-15T19:10:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023597434, 2021-01-15T19:10:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1576 2021-01-15T19:10:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290615+0000", 2021-01-15T19:10:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290615+0000", 2021-01-15T19:10:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4970999999999999e-05 2021-01-15T19:10:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290670+0000", 2021-01-15T19:10:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.4300000000000001e-05 2021-01-15T19:10:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290694+0000", 2021-01-15T19:10:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2883e-05 2021-01-15T19:10:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290707+0000", 2021-01-15T19:10:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9540000000000003e-06 2021-01-15T19:10:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290712+0000", 2021-01-15T19:10:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010065562 2021-01-15T19:10:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.300777+0000", 2021-01-15T19:10:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011069399999999999 2021-01-15T19:10:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.300888+0000", 2021-01-15T19:10:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00053546300000000002 2021-01-15T19:10:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.301423+0000", 2021-01-15T19:10:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010810290000000001 2021-01-15T19:10:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.302504+0000", 2021-01-15T19:10:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011607589 2021-01-15T19:10:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.314112+0000", 2021-01-15T19:10:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9988999999999994e-05 2021-01-15T19:10:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.314212+0000", 2021-01-15T19:10:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023597434 2021-01-15T19:10:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1577 3.10s0 3:0c50693c:::smithi00140323-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e157)", 2021-01-15T19:10:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:25.290753+0000", 2021-01-15T19:10:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.97603954000000004, 2021-01-15T19:10:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025737924999999998, 2021-01-15T19:10:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1577 2021-01-15T19:10:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290753+0000", 2021-01-15T19:10:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290753+0000", 2021-01-15T19:10:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.773e-05 2021-01-15T19:10:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290791+0000", 2021-01-15T19:10:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4219999999999997e-06 2021-01-15T19:10:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290797+0000", 2021-01-15T19:10:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4466e-05 2021-01-15T19:10:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290812+0000", 2021-01-15T19:10:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1259999999999997e-06 2021-01-15T19:10:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.290817+0000", 2021-01-15T19:10:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010929652 2021-01-15T19:10:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.301747+0000", 2021-01-15T19:10:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4047000000000005e-05 2021-01-15T19:10:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.301811+0000", 2021-01-15T19:10:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012875284000000001 2021-01-15T19:10:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.314686+0000", 2021-01-15T19:10:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115196 2021-01-15T19:10:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.314801+0000", 2021-01-15T19:10:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0016900019999999999 2021-01-15T19:10:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:25.316491+0000", 2021-01-15T19:10:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025737924999999998 2021-01-15T19:10:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:26.570 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:26.571 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:26.571 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:10:26.572 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:26.572 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2021-01-15T19:10:26.572 INFO:teuthology.orchestra.run:waiting for 300 2021-01-15T19:10:26.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:26.574 INFO:tasks.rados.rados.0.smithi001.stdout:655: done (2 left) 2021-01-15T19:10:26.575 INFO:tasks.rados.rados.0.smithi001.stdout:656: snap_create 2021-01-15T19:10:26.575 INFO:tasks.rados.rados.0.smithi001.stdout:653: finishing copy_from to smithi00140323-15 2021-01-15T19:10:26.575 INFO:tasks.rados.rados.0.smithi001.stdout:653: got expected ENOENT (src dne) 2021-01-15T19:10:26.575 INFO:tasks.rados.rados.0.smithi001.stdout:653: finishing copy_from racing read to smithi00140323-15 2021-01-15T19:10:26.575 INFO:tasks.rados.rados.0.smithi001.stdout:654: finishing write tid 1 to smithi00140323-16 2021-01-15T19:10:26.576 INFO:tasks.rados.rados.0.smithi001.stdout:654: finishing write tid 2 to smithi00140323-16 2021-01-15T19:10:26.576 INFO:tasks.rados.rados.0.smithi001.stdout:654: finishing write tid 3 to smithi00140323-16 2021-01-15T19:10:26.576 INFO:tasks.rados.rados.0.smithi001.stdout:654: finishing write tid 4 to smithi00140323-16 2021-01-15T19:10:26.576 INFO:tasks.rados.rados.0.smithi001.stdout:654: finishing write tid 5 to smithi00140323-16 2021-01-15T19:10:26.576 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 166 (ObjNum 232 snap 71 seq_num 232) dirty exists 2021-01-15T19:10:26.577 INFO:tasks.rados.rados.0.smithi001.stdout:654: left oid 16 (ObjNum 232 snap 71 seq_num 232) 2021-01-15T19:10:26.577 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:26.571+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:26.612 INFO:tasks.ceph.osd.1:Stopped 2021-01-15T19:10:26.612 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2021-01-15T19:10:26.612 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:10:26.613 DEBUG:teuthology.orchestra.run.smithi001:> type ceph-objectstore-tool 2021-01-15T19:10:26.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:26.667 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2021-01-15T19:10:26.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2021-01-15T19:10:26.673 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:26.672+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:26.774 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:26.773+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:26.790 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:26.791 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:26.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:26.825 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:26.875 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:26.874+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:26.960 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:26.961 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:26.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:26.976 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:26.975+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:27.077 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:27.076+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:27.113 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.86213812299999, 2021-01-15T19:10:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:10:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:10:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:10:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:10:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:10:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:10:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:10:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:10:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:10:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:10:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:10:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:10:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:10:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:10:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:10:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:10:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:10:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:10:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:10:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:10:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:10:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:10:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:10:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.861947179, 2021-01-15T19:10:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:10:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:10:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:10:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:10:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:10:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:10:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:10:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:10:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:10:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:10:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:10:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:10:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:10:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:10:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:10:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:10:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:10:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:10:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:10:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:10:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:10:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:10:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:10:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:10:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:10:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.922995082, 2021-01-15T19:10:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:10:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:10:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:10:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:10:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:10:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:10:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:10:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:10:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:10:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:10:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:10:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:10:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:10:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:10:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:10:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:10:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:10:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:10:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:10:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:10:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:10:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:10:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:10:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.920274621, 2021-01-15T19:10:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:10:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:10:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:10:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:10:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:10:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:10:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:10:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:10:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:10:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:10:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:10:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:10:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:10:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:10:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:10:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:10:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:10:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:10:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:10:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:10:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:10:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:10:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:10:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:10:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:10:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.920162042, 2021-01-15T19:10:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:10:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:10:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:10:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:10:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:10:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:10:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:10:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:10:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:10:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:10:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:10:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:10:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:10:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:10:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:10:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:10:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:10:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:10:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:10:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:10:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:10:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:10:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:10:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.920032138, 2021-01-15T19:10:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:10:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:10:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:10:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:10:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:10:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:10:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:10:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:10:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:10:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:10:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:10:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:10:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:10:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:10:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:10:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:10:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:10:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:10:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:10:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:10:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:10:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:10:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:10:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:10:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:10:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.89682686899999, 2021-01-15T19:10:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:10:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:10:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:10:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:10:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:10:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:10:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:10:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:10:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:10:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:10:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:10:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:10:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:10:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:10:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:10:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:10:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:10:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:10:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:10:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:10:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:10:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.896708671, 2021-01-15T19:10:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:10:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:10:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:10:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:10:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:10:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:10:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:10:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:10:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:10:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:10:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:10:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:10:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:10:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:10:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:10:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:10:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:10:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:10:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:10:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:10:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:10:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:10:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:10:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.896595981, 2021-01-15T19:10:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:10:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:10:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:10:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:10:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:10:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:10:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:10:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:10:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:10:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:10:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:10:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:10:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:10:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:10:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:10:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:10:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:10:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:10:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:10:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:10:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:10:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:10:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:10:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:10:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.89099423099999, 2021-01-15T19:10:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:10:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:10:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:10:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:10:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:10:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:10:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:10:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:10:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:10:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:10:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:10:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:10:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:10:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:10:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.882525507, 2021-01-15T19:10:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:10:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:10:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:10:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:10:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:10:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:10:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:10:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:10:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:10:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:10:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:10:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:10:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:10:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:10:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:10:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:10:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:10:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:10:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:10:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:10:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:10:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:10:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:10:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.878006924, 2021-01-15T19:10:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:10:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:10:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:10:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:10:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:10:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:10:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:10:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:10:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:10:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:10:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:10:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:10:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:10:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:10:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:10:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:10:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:10:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:10:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:10:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:10:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:10:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:10:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:10:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.872989155, 2021-01-15T19:10:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:10:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:10:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:10:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:10:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:10:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:10:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:10:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:10:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:10:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:10:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:10:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:10:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:10:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:10:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:10:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:10:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:10:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:10:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:10:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:10:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:10:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=172, last_complete=56'50, committed=1, applied=1))", 2021-01-15T19:10:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:10:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.864540015, 2021-01-15T19:10:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689, 2021-01-15T19:10:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:10:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246491+0000", 2021-01-15T19:10:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0238999999999999e-05 2021-01-15T19:10:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246531+0000", 2021-01-15T19:10:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.674e-06 2021-01-15T19:10:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246538+0000", 2021-01-15T19:10:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0995999999999999e-05 2021-01-15T19:10:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246549+0000", 2021-01-15T19:10:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.839e-06 2021-01-15T19:10:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.246553+0000", 2021-01-15T19:10:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023113813 2021-01-15T19:10:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269667+0000", 2021-01-15T19:10:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5128e-05 2021-01-15T19:10:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269682+0000", 2021-01-15T19:10:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023190689 2021-01-15T19:10:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:10:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 88.905865077000001, 2021-01-15T19:10:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:10:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:10:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:10:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:10:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:10:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:10:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:10:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:10:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:10:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:10:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:10:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:10:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:10:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:10:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:10:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:10:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:10:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:10:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:10:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:10:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:10:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:10:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:10:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:10:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:10:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:10:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:10:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:10:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 78.928678429000001, 2021-01-15T19:10:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:10:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:10:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:10:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:10:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:10:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:10:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:10:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:10:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:10:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:10:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:10:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:10:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:10:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:10:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:10:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:10:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:10:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:10:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:10:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:10:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:10:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:10:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:10:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:10:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:10:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:10:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:10:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.898131408000001, 2021-01-15T19:10:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:10:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:10:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:10:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:10:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:10:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:10:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:10:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:10:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:10:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:10:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:10:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:10:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:10:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:10:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:10:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:10:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:10:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:10:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:10:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:10:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:10:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:10:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:10:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:10:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:10:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.907473433, 2021-01-15T19:10:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:10:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:10:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:10:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:10:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:10:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:10:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:10:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:10:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:10:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:10:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:10:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:10:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:10:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:10:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:10:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:10:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:10:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:10:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:10:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:10:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:10:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:10:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:10:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:10:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:10:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:10:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.907333900000001, 2021-01-15T19:10:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:10:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:10:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:10:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:10:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:10:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:10:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:10:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:10:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:10:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:10:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:10:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:10:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:10:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:10:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:10:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:10:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:10:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:10:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:10:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:10:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:10:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:10:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:10:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:10:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:10:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:10:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:10:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.907218586999999, 2021-01-15T19:10:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:10:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:10:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:10:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:10:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:10:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:10:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:10:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:10:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:10:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:10:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:10:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:10:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:10:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:10:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:10:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:10:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:10:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:10:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:10:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:10:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:10:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:10:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:10:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:10:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:10:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:10:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:27.433 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:27.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:10:27.436 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:27.436 INFO:tasks.rados.rados.0.smithi001.stdout:653: done (2 left) 2021-01-15T19:10:27.437 INFO:tasks.rados.rados.0.smithi001.stdout:654: done (1 left) 2021-01-15T19:10:27.437 INFO:tasks.rados.rados.0.smithi001.stdout:656: done (0 left) 2021-01-15T19:10:27.437 INFO:tasks.rados.rados.0.smithi001.stdout:657: rollback oid 13 current snap is 72 2021-01-15T19:10:27.437 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 13 to 40 2021-01-15T19:10:27.437 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 36 current snap is 72 2021-01-15T19:10:27.438 INFO:tasks.rados.rados.0.smithi001.stdout:658: seq_num 233 ranges {3325952=712704,4038656=548864,4587520=253952} 2021-01-15T19:10:27.438 INFO:tasks.rados.rados.0.smithi001.stdout:658: writing smithi00140323-36 from 3325952 to 4038656 tid 1 2021-01-15T19:10:27.438 INFO:tasks.rados.rados.0.smithi001.stdout:658: writing smithi00140323-36 from 4038656 to 4587520 tid 2 2021-01-15T19:10:27.438 INFO:tasks.rados.rados.0.smithi001.stdout:658: writing smithi00140323-36 from 4587520 to 4841472 tid 3 2021-01-15T19:10:27.438 INFO:tasks.rados.rados.0.smithi001.stdout:659: snap_create 2021-01-15T19:10:27.439 INFO:tasks.rados.rados.0.smithi001.stdout:657: finishing rollback tid 0 to smithi00140323-13 2021-01-15T19:10:27.439 INFO:tasks.rados.rados.0.smithi001.stdout:657: finishing rollback tid 1 to smithi00140323-13 2021-01-15T19:10:27.439 INFO:tasks.rados.rados.0.smithi001.stdout:657: finishing rollback tid 2 to smithi00140323-13 2021-01-15T19:10:27.439 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 308 (ObjNum 120 snap 27 seq_num 120) dirty exists 2021-01-15T19:10:27.440 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:27.433+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:27.535 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:27.534+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:27.572 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:27.573 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:27.581 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:27.635 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:27.634+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:27.736 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:27.736+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:27.746 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:27.746 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:27.746 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:27.746 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:27.747 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:27.756 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:27.838 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:27.836+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:27.923 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:27.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:10:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.739407510000007, 2021-01-15T19:10:27.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:10:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:27.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:10:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:27.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:10:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:10:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:10:27.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:10:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:10:27.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:10:27.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:10:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:10:27.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:10:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:10:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:10:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:10:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:10:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:10:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:10:27.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:27.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:10:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:10:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:27.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:10:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:10:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:27.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.736824744000003, 2021-01-15T19:10:27.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:10:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:27.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:10:27.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.950 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:27.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:27.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:10:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:10:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:10:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:10:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:10:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:10:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:10:27.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:10:27.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:10:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:27.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:10:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:10:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:10:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:10:27.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:10:27.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:10:27.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:27.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:27.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.736661803000004, 2021-01-15T19:10:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:10:27.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:27.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:10:27.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:27.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:10:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:10:27.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:10:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:27.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:10:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:10:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:27.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:10:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:10:27.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:10:27.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:10:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:10:27.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:10:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:27.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:10:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:10:27.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:27.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:10:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:10:27.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:27.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:27.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.575619157000006, 2021-01-15T19:10:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:10:27.989 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:27.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:27.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:27.992 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:10:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.993 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:27.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:27.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:27.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:27.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:10:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:27.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:10:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:10:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:10:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:10:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:10:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:10:28.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:10:28.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:10:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:10:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:10:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:10:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:10:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:10:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:10:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:10:28.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:10:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:10:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:10:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:10:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:10:28.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:10:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:10:28.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:10:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:10:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:28.019 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.575511700999996, 2021-01-15T19:10:28.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:10:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:10:28.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:10:28.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:10:28.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:10:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:10:28.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:10:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:10:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:10:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:10:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:10:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:10:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:10:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:10:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:10:28.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:10:28.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:10:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:10:28.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:10:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:10:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:10:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:10:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:10:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:10:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:10:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:10:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:10:28.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:10:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:10:28.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.045 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:28.046 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.442444835000003, 2021-01-15T19:10:28.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:10:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.048 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.049 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:10:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:10:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:10:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:10:28.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:10:28.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:10:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:10:28.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:10:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:10:28.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:10:28.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:10:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:10:28.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:10:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:10:28.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:10:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:10:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:10:28.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:10:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:10:28.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:10:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:10:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:10:28.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:10:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:10:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:10:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:10:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.079 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:28.081 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.546477151999994, 2021-01-15T19:10:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:10:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.082 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:10:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:10:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:10:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:10:28.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:10:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:10:28.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:10:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:10:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:10:28.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:10:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:10:28.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:10:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:10:28.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:10:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:10:28.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:10:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:28.104 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.546347022000006, 2021-01-15T19:10:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:10:28.105 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.106 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.107 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:10:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:10:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:10:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:10:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:10:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:10:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:10:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:10:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:10:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:10:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:10:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:10:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:10:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:10:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:10:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:10:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:10:28.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 70.546261612999999, 2021-01-15T19:10:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:10:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:28.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:28.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:28.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:10:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:10:28.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:10:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:10:28.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:10:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:10:28.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:10:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:10:28.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:10:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:10:28.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:10:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:10:28.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:10:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:10:28.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:10:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:10:28.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:10:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:10:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.715087225000005, 2021-01-15T19:10:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:10:28.134 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.135 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:10:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:10:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:10:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:10:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:10:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:10:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:10:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:10:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:10:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:10:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:10:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:10:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:10:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:10:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:10:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:10:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:10:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:10:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:10:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:10:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:10:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:10:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:10:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:10:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:10:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:10:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:10:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:10:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:10:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:10:28.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.153 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:10:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.714998575999999, 2021-01-15T19:10:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:10:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.154 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:10:28.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:10:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:10:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:10:28.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:10:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:28.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:10:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:10:28.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:10:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:10:28.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:10:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:10:28.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:10:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:10:28.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:10:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:10:28.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:10:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:10:28.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:10:28.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:10:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:10:28.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:10:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:10:28.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:10:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:10:28.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:10:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:10:28.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:10:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:10:28.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:10:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.178 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:10:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:28.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.731490518000001, 2021-01-15T19:10:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:10:28.181 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.182 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.183 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:10:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:10:28.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:10:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:10:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:10:28.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:10:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:10:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:10:28.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:10:28.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:10:28.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:10:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:10:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:10:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:10:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:10:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:10:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:10:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:10:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:10:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:10:28.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:10:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:10:28.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:10:28.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:10:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:10:28.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:10:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:10:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.731358442999998, 2021-01-15T19:10:28.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:10:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.206 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:10:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:10:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:10:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:10:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:10:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:10:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:10:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:10:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:10:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:10:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:10:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:10:28.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:10:28.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:10:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:10:28.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:10:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:10:28.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:10:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:10:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:10:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:28.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:10:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:10:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:10:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:10:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:10:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:10:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:10:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:10:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:10:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.728222926000001, 2021-01-15T19:10:28.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:10:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.229 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:10:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:10:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:10:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:10:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:10:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:10:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:10:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:10:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:10:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:10:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:10:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:10:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:10:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:10:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:10:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:10:28.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:10:28.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:10:28.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:10:28.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:10:28.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:10:28.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:10:28.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:10:28.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:10:28.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:10:28.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:10:28.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.249 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.250 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:10:28.250 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:28.251 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.728118899999998, 2021-01-15T19:10:28.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:10:28.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.252 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:10:28.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:28.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:28.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:10:28.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:10:28.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:10:28.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:10:28.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:10:28.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:10:28.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:10:28.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:10:28.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:10:28.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:10:28.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:10:28.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:10:28.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:10:28.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:10:28.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:10:28.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:28.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:10:28.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:10:28.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:10:28.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:10:28.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:28.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:10:28.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:10:28.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:10:28.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:10:28.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:10:28.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:10:28.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:10:28.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:10:28.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:28.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.717132151000001, 2021-01-15T19:10:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:10:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.271 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:10:28.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.271 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:10:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:10:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:10:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:10:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:10:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:10:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:10:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:10:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:10:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:10:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:10:28.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:10:28.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:10:28.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:10:28.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:10:28.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:10:28.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:10:28.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:10:28.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:28.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:28.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.717036629999999, 2021-01-15T19:10:28.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:10:28.285 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.285 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.286 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.286 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:10:28.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.286 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:28.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:28.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:10:28.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:10:28.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:10:28.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:10:28.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:10:28.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:10:28.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:10:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:10:28.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:10:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:10:28.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:10:28.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:10:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:10:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:10:28.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:10:28.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:10:28.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:10:28.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:10:28.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:10:28.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:28.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:28.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.716925484000001, 2021-01-15T19:10:28.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:10:28.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.301 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:10:28.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.302 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:28.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:10:28.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:10:28.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:10:28.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:10:28.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:10:28.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:10:28.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:10:28.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:28.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:10:28.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:10:28.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:10:28.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:10:28.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:10:28.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:10:28.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:10:28.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:10:28.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:10:28.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:10:28.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:10:28.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.315 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:10:28.315 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:28.316 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.62515116199999998, 2021-01-15T19:10:28.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:10:28.316 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.316 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.316 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.317 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.317 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:10:28.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.317 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:28.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:28.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:10:28.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:10:28.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:10:28.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:10:28.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:10:28.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:10:28.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:10:28.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:10:28.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:10:28.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:10:28.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:10:28.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:10:28.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:10:28.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:10:28.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:10:28.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:10:28.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:10:28.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:10:28.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:10:28.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:10:28.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:10:28.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.335 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.336 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:10:28.336 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:28.336 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.62386802299999999, 2021-01-15T19:10:28.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:10:28.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.338 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:10:28.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.338 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:28.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:28.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:10:28.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:10:28.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:10:28.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:10:28.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:10:28.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:10:28.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:10:28.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:10:28.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:10:28.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:10:28.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:10:28.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:10:28.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:10:28.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:10:28.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:10:28.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:10:28.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:10:28.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:10:28.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:10:28.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:10:28.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:10:28.350 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.350 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.351 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.351 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:28.354 INFO:teuthology.orchestra.run.smithi001.stdout:3.fs1 2021-01-15T19:10:28.354 INFO:teuthology.orchestra.run.smithi001.stdout:3.as1 2021-01-15T19:10:28.354 INFO:teuthology.orchestra.run.smithi001.stdout:3.17s2 2021-01-15T19:10:28.354 INFO:teuthology.orchestra.run.smithi001.stdout:3.15s1 2021-01-15T19:10:28.355 INFO:teuthology.orchestra.run.smithi001.stdout:3.14s2 2021-01-15T19:10:28.355 INFO:teuthology.orchestra.run.smithi001.stdout:3.12s0 2021-01-15T19:10:28.355 INFO:teuthology.orchestra.run.smithi001.stdout:3.19s2 2021-01-15T19:10:28.355 INFO:teuthology.orchestra.run.smithi001.stdout:3.11s2 2021-01-15T19:10:28.355 INFO:teuthology.orchestra.run.smithi001.stdout:3.10s0 2021-01-15T19:10:28.356 INFO:teuthology.orchestra.run.smithi001.stdout:2.f 2021-01-15T19:10:28.356 INFO:teuthology.orchestra.run.smithi001.stdout:2.d 2021-01-15T19:10:28.356 INFO:teuthology.orchestra.run.smithi001.stdout:2.7 2021-01-15T19:10:28.356 INFO:teuthology.orchestra.run.smithi001.stdout:2.2 2021-01-15T19:10:28.356 INFO:teuthology.orchestra.run.smithi001.stdout:1.e 2021-01-15T19:10:28.357 INFO:teuthology.orchestra.run.smithi001.stdout:1.d 2021-01-15T19:10:28.357 INFO:teuthology.orchestra.run.smithi001.stdout:1.c 2021-01-15T19:10:28.357 INFO:teuthology.orchestra.run.smithi001.stdout:1.b 2021-01-15T19:10:28.357 INFO:teuthology.orchestra.run.smithi001.stdout:3.7s2 2021-01-15T19:10:28.357 INFO:teuthology.orchestra.run.smithi001.stdout:1.a 2021-01-15T19:10:28.358 INFO:teuthology.orchestra.run.smithi001.stdout:1.f 2021-01-15T19:10:28.358 INFO:teuthology.orchestra.run.smithi001.stdout:1.9 2021-01-15T19:10:28.358 INFO:teuthology.orchestra.run.smithi001.stdout:1.8 2021-01-15T19:10:28.358 INFO:teuthology.orchestra.run.smithi001.stdout:3.6s2 2021-01-15T19:10:28.358 INFO:teuthology.orchestra.run.smithi001.stdout:1.7 2021-01-15T19:10:28.359 INFO:teuthology.orchestra.run.smithi001.stdout:1.3 2021-01-15T19:10:28.359 INFO:teuthology.orchestra.run.smithi001.stdout:3.4s1 2021-01-15T19:10:28.359 INFO:teuthology.orchestra.run.smithi001.stdout:1.1 2021-01-15T19:10:28.359 INFO:teuthology.orchestra.run.smithi001.stdout:1.13 2021-01-15T19:10:28.359 INFO:teuthology.orchestra.run.smithi001.stdout:1.11 2021-01-15T19:10:28.360 INFO:teuthology.orchestra.run.smithi001.stdout:1.10 2021-01-15T19:10:28.360 INFO:teuthology.orchestra.run.smithi001.stdout:3.ds2 2021-01-15T19:10:28.360 INFO:teuthology.orchestra.run.smithi001.stdout:1.0 2021-01-15T19:10:28.360 INFO:tasks.rados.rados.0.smithi001.stdout:657: done (2 left) 2021-01-15T19:10:28.360 INFO:tasks.rados.rados.0.smithi001.stdout:659: done (1 left) 2021-01-15T19:10:28.361 INFO:tasks.rados.rados.0.smithi001.stdout:660: snap_create 2021-01-15T19:10:28.361 INFO:tasks.rados.rados.0.smithi001.stdout:658: finishing write tid 1 to smithi00140323-36 2021-01-15T19:10:28.361 INFO:tasks.rados.rados.0.smithi001.stdout:658: finishing write tid 2 to smithi00140323-36 2021-01-15T19:10:28.361 INFO:tasks.rados.rados.0.smithi001.stdout:658: finishing write tid 3 to smithi00140323-36 2021-01-15T19:10:28.362 INFO:tasks.rados.rados.0.smithi001.stdout:658: finishing write tid 4 to smithi00140323-36 2021-01-15T19:10:28.362 INFO:tasks.rados.rados.0.smithi001.stdout:658: finishing write tid 5 to smithi00140323-36 2021-01-15T19:10:28.362 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 36 v 186 (ObjNum 233 snap 72 seq_num 233) dirty exists 2021-01-15T19:10:28.362 INFO:tasks.rados.rados.0.smithi001.stdout:658: left oid 36 (ObjNum 233 snap 72 seq_num 233) 2021-01-15T19:10:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:10:28.363 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:28.351+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:28.363 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:28.454 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:28.453+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:28.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2021-01-15T19:10:28.520 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:28.521 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:28.521 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:28.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:28.571 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:28.570+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:28.656 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:28.654+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:28.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 3.11s2 --file /var/log/ceph/exp.3.11s2.1 2021-01-15T19:10:28.693 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:28.694 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:28.695 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:28.703 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:28.762 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:28.760+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:28.830 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:28.830 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:10:28.831 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 124.54128226, 2021-01-15T19:10:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:10:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:10:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.833 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:28.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:28.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:10:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:10:28.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:10:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:10:28.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:10:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:10:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:10:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:10:28.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:10:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:10:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:10:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:10:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:10:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:10:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:10:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:10:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:10:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:10:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:10:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:10:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:10:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:10:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:10:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:10:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:10:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.849 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:28.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.609519494, 2021-01-15T19:10:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:10:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.851 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.852 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:10:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:10:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:10:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:10:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:10:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:10:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:10:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:10:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:10:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:10:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:10:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:10:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:10:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:10:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:10:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:10:28.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:10:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:10:28.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:10:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:10:28.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:10:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:10:28.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:10:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:10:28.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:10:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:10:28.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.868 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.607231453, 2021-01-15T19:10:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:10:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.869 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.870 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:10:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:10:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:10:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:10:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:10:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:10:28.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:10:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:10:28.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:10:28.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:10:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:10:28.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:10:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:10:28.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:10:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:10:28.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:10:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:10:28.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:10:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:10:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:10:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:10:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:10:28.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.885 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.604641828, 2021-01-15T19:10:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:10:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.887 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:10:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:10:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:10:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:10:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:10:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:10:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:10:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:10:28.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:10:28.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:10:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:10:28.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:10:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:10:28.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:10:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:10:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:10:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:10:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:10:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:10:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:10:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:10:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:10:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:10:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:28.902 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.60373355, 2021-01-15T19:10:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:10:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.903 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.904 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:10:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:10:28.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:10:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:10:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:10:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:10:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:10:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:10:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:10:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:10:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:10:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:10:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:10:28.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:10:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:10:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:10:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:10:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:10:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:10:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:10:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.918 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:10:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:28.919 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.635049643, 2021-01-15T19:10:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:10:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:10:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:10:28.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:10:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:10:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:10:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:10:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:10:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:10:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:10:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:10:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:10:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:10:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:10:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:10:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:10:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:10:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:10:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:10:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:10:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:10:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:10:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:10:28.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:10:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:10:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:10:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:10:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.937 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:10:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.634899906, 2021-01-15T19:10:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:10:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:10:28.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:28.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:10:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:10:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:10:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:10:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:10:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:10:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:10:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:10:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:10:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:10:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:10:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:10:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:10:28.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:10:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:10:28.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:10:28.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:10:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:10:28.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:10:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:10:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:10:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:10:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:10:28.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:10:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:10:28.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:10:28.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:10:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:10:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.634748008, 2021-01-15T19:10:28.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:10:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:10:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:10:28.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:10:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:10:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:10:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:10:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:10:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:10:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:10:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:10:28.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:10:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:10:28.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:10:28.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:10:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:10:28.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:10:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:28.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:10:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:10:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:28.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:10:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:10:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:10:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:10:28.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:10:28.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:10:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:10:28.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:10:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:10:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.63462269599999, 2021-01-15T19:10:28.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:10:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:28.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:10:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:28.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:28.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:10:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:10:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:10:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:10:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:10:28.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:10:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:10:28.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:28.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:10:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:10:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:10:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:10:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:10:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:10:28.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:10:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:10:28.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:28.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:10:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:10:28.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:10:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:10:28.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:28.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:28.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:28.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:10:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:10:29.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:10:29.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:10:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:10:29.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:10:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:10:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:10:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:10:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:29.006 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 113.61379012099999, 2021-01-15T19:10:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:10:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:10:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:10:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:10:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:10:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:10:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:10:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:10:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:10:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:10:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:10:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:10:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:10:29.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:10:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:10:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:10:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:10:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:10:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:10:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:10:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:10:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:10:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:10:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.022 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:10:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 113.607496789, 2021-01-15T19:10:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:10:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:10:29.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:10:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:10:29.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:10:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:10:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:10:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:10:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:10:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:10:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:10:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:10:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:10:29.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:10:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:10:29.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:10:29.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:10:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:10:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:10:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:10:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:10:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.039 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:10:29.040 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.645437276, 2021-01-15T19:10:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:10:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:10:29.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:29.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:10:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:10:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:10:29.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:10:29.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:10:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:10:29.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:10:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:10:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:10:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:10:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:10:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:10:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:10:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:10:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:10:29.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:10:29.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:10:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:10:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:10:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:10:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:10:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:10:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:10:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:10:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:10:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.059 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:29.060 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.583065296, 2021-01-15T19:10:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:10:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.061 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:10:29.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:10:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:10:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:10:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:10:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:10:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:10:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:10:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:10:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:10:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:10:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:10:29.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:10:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:10:29.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:10:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:10:29.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:10:29.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:10:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:10:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:10:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:10:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:10:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.57976279499999, 2021-01-15T19:10:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:10:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.078 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:10:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:10:29.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:10:29.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:10:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:10:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:10:29.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:10:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:10:29.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:10:29.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:10:29.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:10:29.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:10:29.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:10:29.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:10:29.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:10:29.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:10:29.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:10:29.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:10:29.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:10:29.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:10:29.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:10:29.092 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.093 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.093 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.094 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:29.094 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:29.094 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.578440929, 2021-01-15T19:10:29.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:10:29.094 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.095 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.095 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.096 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:10:29.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.096 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:29.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:29.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:10:29.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:10:29.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:10:29.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:10:29.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:10:29.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:10:29.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:10:29.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:10:29.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:10:29.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:10:29.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:10:29.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:10:29.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:10:29.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:10:29.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:10:29.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:10:29.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:10:29.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:10:29.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:10:29.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:10:29.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:10:29.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.110 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.110 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.110 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:29.111 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:29.111 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.578070676, 2021-01-15T19:10:29.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:10:29.111 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.112 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.112 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.112 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:10:29.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.113 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:29.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:29.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:10:29.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:10:29.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:10:29.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:10:29.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:10:29.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:10:29.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:10:29.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:10:29.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:10:29.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:10:29.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:10:29.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:10:29.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:10:29.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:10:29.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:10:29.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:10:29.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:10:29.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:10:29.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:10:29.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:10:29.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:10:29.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.126 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:29.128 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:29.128 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.57773628299999, 2021-01-15T19:10:29.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:10:29.128 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.128 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.129 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.129 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:10:29.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.130 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:29.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:29.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:10:29.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:10:29.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:10:29.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:10:29.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:10:29.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:10:29.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:10:29.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:10:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:10:29.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:10:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:10:29.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:29.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:10:29.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:10:29.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:10:29.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:10:29.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:10:29.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:10:29.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:10:29.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:10:29.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.143 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.143 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:10:29.143 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:29.144 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 95.636475024000006, 2021-01-15T19:10:29.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:10:29.144 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.144 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.145 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:29.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:29.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:10:29.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:10:29.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:10:29.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:10:29.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:10:29.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:10:29.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:10:29.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:10:29.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:10:29.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:10:29.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:10:29.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:29.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:10:29.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:10:29.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:10:29.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:10:29.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:10:29.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:10:29.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:10:29.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:10:29.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.157 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.158 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:10:29.158 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:29.158 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.644112152000005, 2021-01-15T19:10:29.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:10:29.159 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.159 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.159 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.160 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:10:29.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.160 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:29.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:29.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:10:29.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:10:29.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:10:29.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:10:29.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:10:29.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:10:29.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:29.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:10:29.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:10:29.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:10:29.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:10:29.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:29.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:10:29.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:10:29.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:10:29.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:10:29.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:10:29.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:10:29.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:10:29.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:10:29.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:10:29.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:10:29.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:10:29.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:10:29.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:10:29.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:10:29.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.177 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.178 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:10:29.178 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:29.179 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.614117948000001, 2021-01-15T19:10:29.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:10:29.179 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.179 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.179 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.180 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:10:29.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.181 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:29.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:29.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:10:29.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:10:29.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:10:29.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:10:29.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:10:29.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:10:29.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:10:29.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:10:29.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:10:29.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:10:29.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:10:29.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:29.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:10:29.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:10:29.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:10:29.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:10:29.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:10:29.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:10:29.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:10:29.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:10:29.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.193 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.194 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.194 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:29.196 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:29.195+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:29.196 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:29.203 INFO:tasks.rados.rados.0.smithi001.stdout:658: done (1 left) 2021-01-15T19:10:29.203 INFO:tasks.rados.rados.0.smithi001.stdout:660: done (0 left) 2021-01-15T19:10:29.203 INFO:tasks.rados.rados.0.smithi001.stdout:661: snap_remove snap 45 2021-01-15T19:10:29.297 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:29.296+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:29.364 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:29.364 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:29.365 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:29.365 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:29.373 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:29.415 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:29.414+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:29.500 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:29.499+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:29.539 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:29.539 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:29.540 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:29.540 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:29.549 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:29.633 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:29.632+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:29.702 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:29.701+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:29.715 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:10:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 110.528311976, 2021-01-15T19:10:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:10:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:10:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:10:29.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:10:29.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:10:29.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:10:29.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:10:29.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:10:29.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:10:29.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:10:29.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:10:29.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:10:29.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:10:29.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:10:29.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:10:29.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:10:29.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:10:29.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:10:29.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.739 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.740 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:29.741 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:29.741 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.48082379900001, 2021-01-15T19:10:29.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:10:29.742 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.742 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.743 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.744 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:10:29.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:29.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:29.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:10:29.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:10:29.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:10:29.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:10:29.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:10:29.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:10:29.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:29.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:10:29.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:10:29.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:10:29.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:10:29.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:10:29.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:10:29.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:10:29.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:10:29.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:10:29.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:10:29.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:10:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:10:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:10:29.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:10:29.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.759 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:10:29.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:29.761 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.480675781, 2021-01-15T19:10:29.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:10:29.761 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.763 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:10:29.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.763 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:29.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:29.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:10:29.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:10:29.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:10:29.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:10:29.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:10:29.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:10:29.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:10:29.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:10:29.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:10:29.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:10:29.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:10:29.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:29.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:10:29.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:10:29.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:10:29.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:10:29.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:29.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:10:29.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:10:29.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:10:29.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:10:29.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:10:29.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:10:29.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:10:29.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:10:29.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.779 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.780 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.12s2 85/84 ECSubRead(tid=202, to_read={3:4e4266df:::smithi00140323-37:head=0,2228224,0}, subchunks={3:4e4266df:::smithi00140323-37:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:10:29.780 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:10:29.780 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.360124982000002, 2021-01-15T19:10:29.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293, 2021-01-15T19:10:29.781 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.781 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:29.781 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:10:29.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352908+0000", 2021-01-15T19:10:29.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1519e-05 2021-01-15T19:10:29.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352950+0000", 2021-01-15T19:10:29.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3610000000000001e-06 2021-01-15T19:10:29.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352958+0000", 2021-01-15T19:10:29.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4105e-05 2021-01-15T19:10:29.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352972+0000", 2021-01-15T19:10:29.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3709999999999999e-06 2021-01-15T19:10:29.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.352978+0000", 2021-01-15T19:10:29.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013549736999999999 2021-01-15T19:10:29.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.366527+0000", 2021-01-15T19:10:29.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025352 2021-01-15T19:10:29.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.369063+0000", 2021-01-15T19:10:29.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016154293 2021-01-15T19:10:29.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.791 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.792 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs1 85/84 ECSubWrite(tid=261, reqid=client.4418.0:849, at_version=85'130, trim_to=0'0, roll_forward_to=0'0))", 2021-01-15T19:10:29.792 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:10:29.792 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.313526609999997, 2021-01-15T19:10:29.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001, 2021-01-15T19:10:29.793 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.793 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:29.793 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:10:29.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.399507+0000", 2021-01-15T19:10:29.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041353340000000001 2021-01-15T19:10:29.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403642+0000", 2021-01-15T19:10:29.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6962999999999998e-05 2021-01-15T19:10:29.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403679+0000", 2021-01-15T19:10:29.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7178999999999999e-05 2021-01-15T19:10:29.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403696+0000", 2021-01-15T19:10:29.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:29.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403703+0000", 2021-01-15T19:10:29.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6183999999999997e-05 2021-01-15T19:10:29.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403749+0000", 2021-01-15T19:10:29.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5073e-05 2021-01-15T19:10:29.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.403764+0000", 2021-01-15T19:10:29.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012428852000000001 2021-01-15T19:10:29.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416193+0000", 2021-01-15T19:10:29.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1882000000000002e-05 2021-01-15T19:10:29.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.416245+0000", 2021-01-15T19:10:29.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016738471000000001 2021-01-15T19:10:29.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.805 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.805 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:29.806 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:29.806 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.526354789999999, 2021-01-15T19:10:29.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:10:29.806 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.806 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.807 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:29.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:29.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:29.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:10:29.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:10:29.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:10:29.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:10:29.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:10:29.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:10:29.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:10:29.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:10:29.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:10:29.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:10:29.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:10:29.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:29.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:10:29.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:10:29.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:10:29.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:10:29.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:10:29.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:10:29.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:10:29.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:10:29.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.822 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.823 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:29.823 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:29.823 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.526163500999999, 2021-01-15T19:10:29.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:10:29.824 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.824 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.825 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.825 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:29.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.825 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:29.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:29.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:10:29.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:10:29.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:10:29.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:10:29.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:10:29.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:10:29.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:10:29.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:10:29.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:10:29.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:10:29.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:10:29.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:29.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:10:29.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:10:29.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:10:29.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:10:29.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:10:29.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:10:29.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:10:29.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:10:29.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.838 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.838 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.838 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:29.839 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:29.839 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.526001148000006, 2021-01-15T19:10:29.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:10:29.839 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.840 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.840 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:29.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.841 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:29.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:29.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:10:29.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:10:29.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:10:29.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:10:29.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:10:29.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:10:29.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:10:29.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:10:29.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:10:29.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:10:29.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:10:29.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:29.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:10:29.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:10:29.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:10:29.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:10:29.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:10:29.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:10:29.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:10:29.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:10:29.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.852 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.853 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:982 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:29.853 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:10:29.853 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.494341997999996, 2021-01-15T19:10:29.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641, 2021-01-15T19:10:29.854 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.854 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.854 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.855 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 982 2021-01-15T19:10:29.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.855 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:10:29.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218691+0000", 2021-01-15T19:10:29.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8164999999999999e-05 2021-01-15T19:10:29.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218719+0000", 2021-01-15T19:10:29.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6679999999999999e-06 2021-01-15T19:10:29.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218723+0000", 2021-01-15T19:10:29.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0639999999999999e-06 2021-01-15T19:10:29.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218729+0000", 2021-01-15T19:10:29.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5579999999999999e-06 2021-01-15T19:10:29.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218732+0000", 2021-01-15T19:10:29.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033278300000000002 2021-01-15T19:10:29.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219064+0000", 2021-01-15T19:10:29.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118377 2021-01-15T19:10:29.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219183+0000", 2021-01-15T19:10:29.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014915026 2021-01-15T19:10:29.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.234098+0000", 2021-01-15T19:10:29.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015406641 2021-01-15T19:10:29.864 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.864 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.865 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:29.865 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:29.866 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.494281235000003, 2021-01-15T19:10:29.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:10:29.866 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.866 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.867 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.867 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.867 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:10:29.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.867 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:29.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:29.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:10:29.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:10:29.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:10:29.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:10:29.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:10:29.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:10:29.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:10:29.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:10:29.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:10:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:10:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:10:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:10:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:10:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:10:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:10:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.494219545999997, 2021-01-15T19:10:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:10:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:10:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:10:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:10:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:10:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:10:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:10:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:10:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:10:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:10:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:10:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:10:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:10:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:10:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:10:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:10:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:10:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 110/52 ECSubWrite(tid=513, reqid=client.4418.0:1096, at_version=110'199, trim_to=0'0, roll_forward_to=109'197))", 2021-01-15T19:10:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:10:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.495558998999996, 2021-01-15T19:10:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999, 2021-01-15T19:10:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:10:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217474+0000", 2021-01-15T19:10:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091685599999999996 2021-01-15T19:10:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218391+0000", 2021-01-15T19:10:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6659999999999998e-05 2021-01-15T19:10:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218428+0000", 2021-01-15T19:10:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6130999999999999e-05 2021-01-15T19:10:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218444+0000", 2021-01-15T19:10:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.114e-05 2021-01-15T19:10:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218455+0000", 2021-01-15T19:10:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0050000000000003e-05 2021-01-15T19:10:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218515+0000", 2021-01-15T19:10:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4779e-05 2021-01-15T19:10:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.218530+0000", 2021-01-15T19:10:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014431424 2021-01-15T19:10:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.232961+0000", 2021-01-15T19:10:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.893e-05 2021-01-15T19:10:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233020+0000", 2021-01-15T19:10:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015545969999999999 2021-01-15T19:10:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:10:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.390153208000001, 2021-01-15T19:10:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:10:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:10:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:10:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:10:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:10:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:10:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:10:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:10:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:10:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:10:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:10:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:10:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:10:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:10:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:10:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:10:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:10:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:10:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:10:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:10:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:10:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:10:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:10:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:10:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:10:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:10:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:10:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:10:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.390066636, 2021-01-15T19:10:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:10:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:10:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:29.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:29.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:10:29.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:10:29.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:10:29.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:10:29.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:10:29.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:10:29.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:10:29.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:10:29.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:10:29.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:10:29.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:10:29.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:29.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:10:29.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:10:29.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:10:29.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:10:29.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:29.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:10:29.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:10:29.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:10:29.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:10:29.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:29.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:10:29.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:10:29.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:10:29.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:10:29.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:10:29.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:10:29.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:10:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:10:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.8s1 135/52 ECSubRead(tid=613, to_read={3:125ea3c2:::smithi00140323-13:head=0,3346432,0}, subchunks={3:125ea3c2:::smithi00140323-13:head=[0,1]}, attrs_to_read=))", 2021-01-15T19:10:29.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213213+0000", 2021-01-15T19:10:29.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.499820177, 2021-01-15T19:10:29.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015731123, 2021-01-15T19:10:29.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:29.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213213+0000", 2021-01-15T19:10:29.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213213+0000", 2021-01-15T19:10:29.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.34e-05 2021-01-15T19:10:29.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213236+0000", 2021-01-15T19:10:29.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9779999999999996e-06 2021-01-15T19:10:29.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213241+0000", 2021-01-15T19:10:29.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2039999999999997e-06 2021-01-15T19:10:29.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213250+0000", 2021-01-15T19:10:29.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.371e-06 2021-01-15T19:10:29.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213253+0000", 2021-01-15T19:10:29.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01303675 2021-01-15T19:10:29.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.226290+0000", 2021-01-15T19:10:29.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026544200000000002 2021-01-15T19:10:29.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228944+0000", 2021-01-15T19:10:29.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015731123 2021-01-15T19:10:29.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.948 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:10:29.948 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:29.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.499756507000001, 2021-01-15T19:10:29.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:10:29.949 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.949 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:29.949 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:29.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:29.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:10:29.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:10:29.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:10:29.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:10:29.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:10:29.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:10:29.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:10:29.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:10:29.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:10:29.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:10:29.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:10:29.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:10:29.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:10:29.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:10:29.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:10:29.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:10:29.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:10:29.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.960 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:10:29.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:29.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.374595658, 2021-01-15T19:10:29.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:10:29.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.962 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:10:29.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.963 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:29.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:29.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:10:29.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:10:29.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:10:29.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:10:29.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:10:29.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:10:29.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:10:29.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:10:29.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:10:29.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:29.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:10:29.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:10:29.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:10:29.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:29.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:10:29.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:10:29.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:29.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:10:29.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:10:29.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:29.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:10:29.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:10:29.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:10:29.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:10:29.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.975 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.975 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.976 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:10:29.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:29.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.374499738000001, 2021-01-15T19:10:29.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:10:29.977 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.977 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:29.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.978 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.978 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:10:29.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.978 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:29.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:29.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:10:29.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:10:29.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:10:29.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:10:29.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:10:29.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:10:29.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:10:29.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:29.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:10:29.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:10:29.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:10:29.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:10:29.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:29.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:10:29.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:10:29.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:10:29.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:10:29.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:29.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:10:29.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:10:29.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:29.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:10:29.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:10:29.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:29.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:10:29.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:10:29.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:29.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:10:29.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:10:29.991 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:29.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:29.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.992 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:10:29.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:29.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.507441398000001, 2021-01-15T19:10:29.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:10:29.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:29.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:29.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:29.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:29.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:29.994 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:10:29.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.995 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:29.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:29.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:29.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:29.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:29.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:29.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:10:29.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:29.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:10:29.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:10:29.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:29.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:10:29.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:10:29.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:29.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:29.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:29.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:10:29.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:10:30.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:30.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:10:30.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:10:30.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:30.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:10:30.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:10:30.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:30.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:10:30.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:10:30.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:30.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:10:30.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:10:30.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:30.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:10:30.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:10:30.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:30.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:10:30.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:10:30.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:30.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:10:30.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:10:30.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:30.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:10:30.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:10:30.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:30.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:10:30.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:10:30.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:30.009 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:30.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:30.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:10:30.010 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:30.010 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.507350321000001, 2021-01-15T19:10:30.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:10:30.010 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:30.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:30.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:30.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:30.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:30.011 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:10:30.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:30.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:30.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:30.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:30.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:30.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:30.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:10:30.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:30.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:10:30.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:10:30.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:30.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:10:30.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:10:30.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:30.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:10:30.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:10:30.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:30.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:10:30.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:10:30.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:30.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:10:30.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:10:30.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:30.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:10:30.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:10:30.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:30.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:10:30.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:10:30.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:30.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:10:30.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:10:30.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:30.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:10:30.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:10:30.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:30.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:10:30.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:10:30.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:10:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:10:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:10:30.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:10:30.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:30.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:30.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:30.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:10:30.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:10:30.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:30.027 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:30.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:30.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:30.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:30.028 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:Exporting 3.11s2 info 3.11s2( v 157'116 (0'0,157'116] local-lis/les=122/123 n=6 ec=52/21 lis/c=122/122 les/c/f=123/123/0 sis=122) 2021-01-15T19:10:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8b30e322:::smithi00140323-24:50# 2021-01-15T19:10:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8b30e322:::smithi00140323-24:6e# 2021-01-15T19:10:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8b30e322:::smithi00140323-24:head# 2021-01-15T19:10:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8f2f8304:::smithi00140323-26:53# 2021-01-15T19:10:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8f2f8304:::smithi00140323-26:67# 2021-01-15T19:10:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8f2f8304:::smithi00140323-26:6d# 2021-01-15T19:10:30.032 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:8f2f8304:::smithi00140323-26:head# 2021-01-15T19:10:30.032 INFO:teuthology.orchestra.run.smithi001.stderr:Export successful 2021-01-15T19:10:30.032 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:30.028+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:30.033 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:30.131 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:30.129+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:30.196 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:30.196 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:30.197 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:30.197 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:30.203 INFO:tasks.rados.rados.0.smithi001.stdout:661: done (0 left) 2021-01-15T19:10:30.203 INFO:tasks.rados.rados.0.smithi001.stdout:662: setattr oid 22 current snap is 74 2021-01-15T19:10:30.204 INFO:tasks.rados.rados.0.smithi001.stdout:663: read oid 30 snap -1 2021-01-15T19:10:30.204 INFO:tasks.rados.rados.0.smithi001.stdout:663: expect deleted 2021-01-15T19:10:30.205 INFO:tasks.rados.rados.0.smithi001.stdout:664: snap_create 2021-01-15T19:10:30.211 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:30.214 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 22 v 170 (ObjNum 158 snap 41 seq_num 158) dirty exists 2021-01-15T19:10:30.254 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:30.253+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:30.333 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:30.331+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:30.336 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:30.337 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:30.337 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:30.338 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:30.338 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:30.346 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:30.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:10:30.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 3.11s2 2021-01-15T19:10:30.444 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:30.442+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:30.512 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:30.512 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:30.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:30.513 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:30.513 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:30.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:30.534 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:30.534+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:30.636 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:30.634+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:30.657 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:30.657 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:30.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:30.751 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:30.750+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:30.838 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:30.837+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:30.844 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:30.844 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:30.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:30.939 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:30.938+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:31.026 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:31.026 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:31.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:31.041 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:31.040+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:31.142 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:31.141+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:31.167 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:31.167 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:31.168 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:31.168 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:31.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:31.206 INFO:tasks.rados.rados.0.smithi001.stdout:662: done (2 left) 2021-01-15T19:10:31.207 INFO:tasks.rados.rados.0.smithi001.stdout:663: done (1 left) 2021-01-15T19:10:31.207 INFO:tasks.rados.rados.0.smithi001.stdout:664: done (0 left) 2021-01-15T19:10:31.207 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 18 current snap is 75 2021-01-15T19:10:31.207 INFO:tasks.rados.rados.0.smithi001.stdout:665: seq_num 234 ranges {1425408=729088,2154496=434176,2588672=368640} 2021-01-15T19:10:31.211 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:10:31.218 INFO:tasks.rados.rados.0.smithi001.stdout:665: writing smithi00140323-18 from 1425408 to 2154496 tid 1 2021-01-15T19:10:31.221 INFO:tasks.rados.rados.0.smithi001.stdout:665: writing smithi00140323-18 from 2154496 to 2588672 tid 2 2021-01-15T19:10:31.223 INFO:tasks.rados.rados.0.smithi001.stdout:665: writing smithi00140323-18 from 2588672 to 2957312 tid 3 2021-01-15T19:10:31.224 INFO:tasks.rados.rados.0.smithi001.stdout:666: delete oid 39 current snap is 75 2021-01-15T19:10:31.230 INFO:tasks.rados.rados.0.smithi001.stdout:666: done (1 left) 2021-01-15T19:10:31.230 INFO:tasks.rados.rados.0.smithi001.stdout:667: delete oid 37 current snap is 75 2021-01-15T19:10:31.235 INFO:tasks.rados.rados.0.smithi001.stdout:665: finishing write tid 1 to smithi00140323-18 2021-01-15T19:10:31.236 INFO:tasks.rados.rados.0.smithi001.stdout:665: finishing write tid 2 to smithi00140323-18 2021-01-15T19:10:31.236 INFO:tasks.rados.rados.0.smithi001.stdout:665: finishing write tid 3 to smithi00140323-18 2021-01-15T19:10:31.238 INFO:tasks.rados.rados.0.smithi001.stdout:665: finishing write tid 4 to smithi00140323-18 2021-01-15T19:10:31.240 INFO:tasks.rados.rados.0.smithi001.stdout:667: done (1 left) 2021-01-15T19:10:31.241 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 6 current snap is 75 2021-01-15T19:10:31.241 INFO:tasks.rados.rados.0.smithi001.stdout:668: seq_num 237 ranges {1105920=778240,1884160=532480,2416640=270336} 2021-01-15T19:10:31.243 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:31.242+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:31.250 INFO:tasks.rados.rados.0.smithi001.stdout:668: writing smithi00140323-6 from 1105920 to 1884160 tid 1 2021-01-15T19:10:31.253 INFO:tasks.rados.rados.0.smithi001.stdout:668: writing smithi00140323-6 from 1884160 to 2416640 tid 2 2021-01-15T19:10:31.254 INFO:tasks.rados.rados.0.smithi001.stdout:668: writing smithi00140323-6 from 2416640 to 2686976 tid 3 2021-01-15T19:10:31.255 INFO:tasks.rados.rados.0.smithi001.stdout:665: finishing write tid 5 to smithi00140323-18 2021-01-15T19:10:31.255 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 18 v 225 (ObjNum 234 snap 75 seq_num 234) dirty exists 2021-01-15T19:10:31.255 INFO:tasks.rados.rados.0.smithi001.stdout:665: left oid 18 (ObjNum 234 snap 75 seq_num 234) 2021-01-15T19:10:31.256 INFO:tasks.rados.rados.0.smithi001.stdout:665: done (1 left) 2021-01-15T19:10:31.256 INFO:tasks.rados.rados.0.smithi001.stdout:669: delete oid 41 current snap is 75 2021-01-15T19:10:31.258 INFO:tasks.rados.rados.0.smithi001.stdout:669: done (1 left) 2021-01-15T19:10:31.259 INFO:tasks.rados.rados.0.smithi001.stdout:670: copy_from oid 19 from oid 32 current snap is 75 2021-01-15T19:10:31.259 INFO:tasks.rados.rados.0.smithi001.stdout:671: delete oid 11 current snap is 75 2021-01-15T19:10:31.262 INFO:tasks.rados.rados.0.smithi001.stdout:668: finishing write tid 1 to smithi00140323-6 2021-01-15T19:10:31.262 INFO:tasks.rados.rados.0.smithi001.stdout:668: finishing write tid 2 to smithi00140323-6 2021-01-15T19:10:31.264 INFO:tasks.rados.rados.0.smithi001.stdout:671: done (2 left) 2021-01-15T19:10:31.264 INFO:tasks.rados.rados.0.smithi001.stdout:672: snap_remove snap 54 2021-01-15T19:10:31.330 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:31.331 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:31.331 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:31.331 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:31.331 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:31.341 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:31.383 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:31.381+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:31.445 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:31.444+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:31.516 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:31.517 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:31.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:31.518 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:31.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.519 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:31.519 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:31.519 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 108.078237798, 2021-01-15T19:10:31.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:10:31.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.521 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:10:31.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.524 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:31.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:31.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:10:31.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:10:31.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:10:31.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:10:31.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:10:31.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:10:31.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:31.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:10:31.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:10:31.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:10:31.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:10:31.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:10:31.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:10:31.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:10:31.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:10:31.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:10:31.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:10:31.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:10:31.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:10:31.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:10:31.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:10:31.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:10:31.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:10:31.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:10:31.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:10:31.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:10:31.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:10:31.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:10:31.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:10:31.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.551 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.552 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:31.552 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:31.553 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 108.078098498, 2021-01-15T19:10:31.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:10:31.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.554 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.554 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.554 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:10:31.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.555 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:31.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:31.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:10:31.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:10:31.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:31.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:10:31.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:10:31.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:10:31.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:10:31.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:10:31.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:10:31.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:10:31.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:10:31.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:10:31.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:10:31.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:10:31.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:10:31.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:10:31.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:10:31.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:10:31.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:10:31.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:10:31.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:10:31.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:10:31.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:10:31.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:10:31.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:10:31.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:10:31.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:10:31.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:10:31.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:10:31.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.573 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.575 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:31.575 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:31.575 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 108.07797739199999, 2021-01-15T19:10:31.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:10:31.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.576 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.576 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.576 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:10:31.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.577 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:31.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:31.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:10:31.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:10:31.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:10:31.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:10:31.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:10:31.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:10:31.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:10:31.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:10:31.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:10:31.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:10:31.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:10:31.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:10:31.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:10:31.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:10:31.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:10:31.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.587 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:10:31.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:10:31.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:10:31.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:10:31.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:10:31.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:10:31.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:10:31.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:10:31.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:10:31.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:10:31.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:10:31.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:10:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:10:31.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:10:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.307696518, 2021-01-15T19:10:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:10:31.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:10:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:10:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:10:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:10:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:10:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:10:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:10:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:10:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:10:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:10:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:10:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:10:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:10:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:10:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:10:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:10:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:10:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:10:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:10:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:10:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:10:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:10:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:10:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:10:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:10:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:10:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.611 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:31.612 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.29871833999999, 2021-01-15T19:10:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:10:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:10:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.614 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:31.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:31.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:10:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:10:31.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:10:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:10:31.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:10:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:10:31.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:10:31.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:10:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:10:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:10:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:10:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:10:31.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:10:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:10:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:10:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:10:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:10:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:10:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:10:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:10:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:10:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:10:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:10:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:31.633 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.29819864, 2021-01-15T19:10:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:10:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.634 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:10:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:10:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:10:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:10:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:10:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:10:31.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:10:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:10:31.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:10:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:10:31.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:10:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:10:31.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:10:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:10:31.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:10:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:10:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:10:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:10:31.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:10:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:10:31.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:10:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:10:31.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:10:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:10:31.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:10:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:10:31.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.650 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.296517686, 2021-01-15T19:10:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:10:31.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:10:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:10:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:10:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:10:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:10:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:10:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:10:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:10:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:10:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:10:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:10:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:10:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:10:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:10:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:10:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:10:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:10:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:10:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:10:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:10:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:10:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:10:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:10:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:10:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:10:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:31.668 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 107.29638473200001, 2021-01-15T19:10:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:10:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:10:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:10:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:10:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:10:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:10:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:10:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:10:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:10:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:10:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:10:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:10:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:10:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:10:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:10:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:10:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:10:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:10:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:10:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:10:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:10:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:10:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:10:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:10:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.683 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:31.684 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 98.333853461999993, 2021-01-15T19:10:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:10:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.685 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.686 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:10:31.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:31.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:31.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:10:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:10:31.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:10:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:10:31.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:10:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:10:31.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:10:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:10:31.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:10:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:10:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:10:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:10:31.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:10:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:10:31.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:10:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:10:31.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:10:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:10:31.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:10:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:10:31.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:10:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:10:31.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:10:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:10:31.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:10:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 98.333686869000005, 2021-01-15T19:10:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:10:31.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:10:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:10:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:10:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:10:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:10:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:10:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:10:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:10:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:10:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:10:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:10:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:10:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:31.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:10:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:10:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:10:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:10:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:31.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:10:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:10:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:10:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:10:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:10:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:10:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:10:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:10:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:10:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:10:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:10:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:10:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.719 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:31.720 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.135148075000004, 2021-01-15T19:10:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:10:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.721 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:10:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:10:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:10:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:10:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:10:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:10:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:10:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:10:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:10:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:10:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:10:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:10:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:10:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:10:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:10:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:10:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:10:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:10:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:10:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:10:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:10:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:10:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:10:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:10:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:10:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:10:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.736 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.737 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.132878030000001, 2021-01-15T19:10:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:10:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:10:31.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:31.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:31.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:10:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:10:31.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:10:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:10:31.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:10:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:10:31.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:10:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:10:31.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:10:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:10:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:10:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:10:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:10:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:10:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:10:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:10:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:10:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:10:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:10:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:10:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:10:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:10:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:10:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:10:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:10:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.128339839000006, 2021-01-15T19:10:31.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:10:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.756 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:10:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:10:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:10:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:10:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:10:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:10:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:10:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:10:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:10:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:10:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:10:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:10:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:10:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:10:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:10:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:10:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:10:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:10:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:10:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:10:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:10:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:10:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:10:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:10:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:10:31.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.772 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.125753966999994, 2021-01-15T19:10:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:10:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:10:31.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:31.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:10:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:10:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:10:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:10:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:10:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:10:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:10:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:10:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:10:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:10:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:10:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:10:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:10:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:10:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:10:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:10:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:10:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:10:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:10:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:10:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:10:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:10:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:10:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:10:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:10:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:31.790 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.125379632000005, 2021-01-15T19:10:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:10:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:10:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:10:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:10:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:10:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:10:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:10:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:10:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:10:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:10:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:10:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:10:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:10:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:10:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:10:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:10:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:10:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:10:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:10:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:10:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:10:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:10:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:10:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:10:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:10:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:10:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:10:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.806 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:31.807 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.125243136999998, 2021-01-15T19:10:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:10:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:31.808 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:10:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.809 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:31.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:31.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:10:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:10:31.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:10:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:10:31.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:10:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:10:31.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:10:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:10:31.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:10:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:10:31.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:10:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:10:31.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:10:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:10:31.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:10:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:10:31.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:10:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:10:31.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:10:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:10:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:10:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:10:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:10:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:10:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:31.824 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 77.142845417000004, 2021-01-15T19:10:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:10:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.825 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:10:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:10:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:10:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:10:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:10:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:10:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:10:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:10:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:10:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:10:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:10:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:10:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:10:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:10:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:10:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:10:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:10:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:10:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:10:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:10:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:10:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:10:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:10:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:10:31.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:10:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9960923590000004, 2021-01-15T19:10:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:10:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:10:31.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:31.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:31.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:10:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:10:31.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:10:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:10:31.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:10:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:10:31.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:10:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:10:31.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:10:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:10:31.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:10:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:10:31.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:10:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:10:31.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:10:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:31.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:10:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:10:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:10:31.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:10:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:10:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:10:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:10:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.9959294270000001, 2021-01-15T19:10:31.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:10:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:31.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:10:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:10:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:10:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:10:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:10:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:10:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:10:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:10:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:10:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:10:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:10:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:10:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:10:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:10:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:10:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:10:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:10:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:10:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:10:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:10:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:10:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:10:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:10:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:10:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:10:31.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 4.2222567900000003, 2021-01-15T19:10:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:10:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:31.873 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:31.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:10:31.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:10:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:10:31.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:10:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:10:31.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:10:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:10:31.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:10:31.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:10:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:10:31.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:10:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:10:31.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:10:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:10:31.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:10:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:10:31.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:10:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:10:31.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:10:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:31.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:31.886 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:31.888 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:31.888 INFO:teuthology.orchestra.run.smithi001.stdout: marking collection for removal 2021-01-15T19:10:31.889 INFO:teuthology.orchestra.run.smithi001.stdout:setting '_remove' omap key 2021-01-15T19:10:31.889 INFO:teuthology.orchestra.run.smithi001.stdout:finish_remove_pgs 3.11s2_head removing 3.11s2 2021-01-15T19:10:31.889 INFO:teuthology.orchestra.run.smithi001.stdout:Remove successful 2021-01-15T19:10:31.889 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:31.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:31.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:31.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:31.890 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:31.886+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:31.904 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:31.927 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:31.927 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.927 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:31.927 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:31.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.928 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.929 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:31.929 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:31.930 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:31.988 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:31.986+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:32.024 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.024 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:32.088 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:32.087+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:32.093 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.11s2.1 2021-01-15T19:10:32.189 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:32.188+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:32.209 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.209 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.220 INFO:tasks.rados.rados.0.smithi001.stdout:672: done (2 left) 2021-01-15T19:10:32.220 INFO:tasks.rados.rados.0.smithi001.stdout:673: read oid 26 snap -1 2021-01-15T19:10:32.221 INFO:tasks.rados.rados.0.smithi001.stdout:673: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:10:32.221 INFO:tasks.rados.rados.0.smithi001.stdout:674: copy_from oid 47 from oid 36 current snap is 75 2021-01-15T19:10:32.221 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 22 current snap is 75 2021-01-15T19:10:32.222 INFO:tasks.rados.rados.0.smithi001.stdout:675: seq_num 240 ranges {933888=425984,1359872=589824,1949696=483328,2433024=131072} 2021-01-15T19:10:32.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:32.228 INFO:tasks.rados.rados.0.smithi001.stdout:675: writing smithi00140323-22 from 933888 to 1359872 tid 1 2021-01-15T19:10:32.232 INFO:tasks.rados.rados.0.smithi001.stdout:675: writing smithi00140323-22 from 1359872 to 1949696 tid 2 2021-01-15T19:10:32.236 INFO:tasks.rados.rados.0.smithi001.stdout:675: writing smithi00140323-22 from 1949696 to 2433024 tid 3 2021-01-15T19:10:32.237 INFO:tasks.rados.rados.0.smithi001.stdout:675: writing smithi00140323-22 from 2433024 to 2564096 tid 4 2021-01-15T19:10:32.238 INFO:tasks.rados.rados.0.smithi001.stdout:668: finishing write tid 3 to smithi00140323-6 2021-01-15T19:10:32.239 INFO:tasks.rados.rados.0.smithi001.stdout:668: finishing write tid 4 to smithi00140323-6 2021-01-15T19:10:32.239 INFO:tasks.rados.rados.0.smithi001.stdout:668: finishing write tid 5 to smithi00140323-6 2021-01-15T19:10:32.240 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 6 v 175 (ObjNum 237 snap 75 seq_num 237) dirty exists 2021-01-15T19:10:32.240 INFO:tasks.rados.rados.0.smithi001.stdout:668: left oid 6 (ObjNum 237 snap 75 seq_num 237) 2021-01-15T19:10:32.240 INFO:tasks.rados.rados.0.smithi001.stdout:670: finishing copy_from to smithi00140323-19 2021-01-15T19:10:32.241 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 149 (ObjNum 206 snap 64 seq_num 206) dirty exists 2021-01-15T19:10:32.241 INFO:tasks.rados.rados.0.smithi001.stdout:670: finishing copy_from racing read to smithi00140323-19 2021-01-15T19:10:32.242 INFO:tasks.rados.rados.0.smithi001.stdout:673: expect (ObjNum 157 snap 41 seq_num 157) 2021-01-15T19:10:32.246 INFO:tasks.rados.rados.0.smithi001.stdout:675: finishing write tid 1 to smithi00140323-22 2021-01-15T19:10:32.247 INFO:tasks.rados.rados.0.smithi001.stdout:675: finishing write tid 2 to smithi00140323-22 2021-01-15T19:10:32.247 INFO:tasks.rados.rados.0.smithi001.stdout:668: done (4 left) 2021-01-15T19:10:32.248 INFO:tasks.rados.rados.0.smithi001.stdout:670: done (3 left) 2021-01-15T19:10:32.248 INFO:tasks.rados.rados.0.smithi001.stdout:673: done (2 left) 2021-01-15T19:10:32.249 INFO:tasks.rados.rados.0.smithi001.stdout:676: rollback oid 4 current snap is 75 2021-01-15T19:10:32.249 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 4 to 64 2021-01-15T19:10:32.249 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 41 current snap is 75 2021-01-15T19:10:32.250 INFO:tasks.rados.rados.0.smithi001.stdout:677: seq_num 241 ranges {0=442368,442368=466944,909312=688128,1597440=49152} 2021-01-15T19:10:32.250 INFO:tasks.rados.rados.0.smithi001.stdout:677: writing smithi00140323-41 from 0 to 442368 tid 1 2021-01-15T19:10:32.252 INFO:tasks.rados.rados.0.smithi001.stdout:677: writing smithi00140323-41 from 442368 to 909312 tid 2 2021-01-15T19:10:32.258 INFO:tasks.rados.rados.0.smithi001.stdout:677: writing smithi00140323-41 from 909312 to 1597440 tid 3 2021-01-15T19:10:32.258 INFO:tasks.rados.rados.0.smithi001.stdout:677: writing smithi00140323-41 from 1597440 to 1646592 tid 4 2021-01-15T19:10:32.259 INFO:tasks.rados.rados.0.smithi001.stdout:675: finishing write tid 3 to smithi00140323-22 2021-01-15T19:10:32.259 INFO:tasks.rados.rados.0.smithi001.stdout:678: read oid 23 snap -1 2021-01-15T19:10:32.259 INFO:tasks.rados.rados.0.smithi001.stdout:678: expect (ObjNum 181 snap 58 seq_num 181) 2021-01-15T19:10:32.260 INFO:tasks.rados.rados.0.smithi001.stdout:679: copy_from oid 32 from oid 3 current snap is 75 2021-01-15T19:10:32.260 INFO:tasks.rados.rados.0.smithi001.stdout:680: copy_from oid 2 from oid 1 current snap is 75 2021-01-15T19:10:32.261 INFO:tasks.rados.rados.0.smithi001.stdout:681: delete oid 9 current snap is 75 2021-01-15T19:10:32.265 INFO:tasks.rados.rados.0.smithi001.stdout:676: finishing rollback tid 0 to smithi00140323-4 2021-01-15T19:10:32.267 INFO:tasks.rados.rados.0.smithi001.stdout:676: finishing rollback tid 1 to smithi00140323-4 2021-01-15T19:10:32.270 INFO:tasks.rados.rados.0.smithi001.stdout:677: finishing write tid 1 to smithi00140323-41 2021-01-15T19:10:32.270 INFO:tasks.rados.rados.0.smithi001.stdout:677: finishing write tid 2 to smithi00140323-41 2021-01-15T19:10:32.271 INFO:tasks.rados.rados.0.smithi001.stdout:676: finishing rollback tid 2 to smithi00140323-4 2021-01-15T19:10:32.272 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 152 (ObjNum 78 snap 13 seq_num 78) dirty exists 2021-01-15T19:10:32.275 INFO:tasks.rados.rados.0.smithi001.stdout:680: finishing copy_from to smithi00140323-2 2021-01-15T19:10:32.275 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 160 (ObjNum 1852461664 snap 1651658338 seq_num 1901091160) dirty exists 2021-01-15T19:10:32.276 INFO:tasks.rados.rados.0.smithi001.stdout:680: finishing copy_from racing read to smithi00140323-2 2021-01-15T19:10:32.281 INFO:tasks.rados.rados.0.smithi001.stdout:678: expect (ObjNum 181 snap 58 seq_num 181) 2021-01-15T19:10:32.290 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:32.289+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:32.382 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.382 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.390 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:32.390+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:32.392 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:32.446 INFO:tasks.rados.rados.0.smithi001.stdout:677: finishing write tid 3 to smithi00140323-41 2021-01-15T19:10:32.446 INFO:tasks.rados.rados.0.smithi001.stdout:677: finishing write tid 4 to smithi00140323-41 2021-01-15T19:10:32.446 INFO:tasks.rados.rados.0.smithi001.stdout:677: finishing write tid 5 to smithi00140323-41 2021-01-15T19:10:32.447 INFO:tasks.rados.rados.0.smithi001.stdout:677: finishing write tid 6 to smithi00140323-41 2021-01-15T19:10:32.447 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 41 v 135 (ObjNum 241 snap 75 seq_num 241) dirty exists 2021-01-15T19:10:32.447 INFO:tasks.rados.rados.0.smithi001.stdout:677: left oid 41 (ObjNum 241 snap 75 seq_num 241) 2021-01-15T19:10:32.447 INFO:tasks.rados.rados.0.smithi001.stdout:679: finishing copy_from to smithi00140323-32 2021-01-15T19:10:32.447 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 32 v 94 (ObjNum 231 snap 71 seq_num 231) dirty exists 2021-01-15T19:10:32.448 INFO:tasks.rados.rados.0.smithi001.stdout:675: finishing write tid 4 to smithi00140323-22 2021-01-15T19:10:32.448 INFO:tasks.rados.rados.0.smithi001.stdout:675: finishing write tid 5 to smithi00140323-22 2021-01-15T19:10:32.448 INFO:tasks.rados.rados.0.smithi001.stdout:679: finishing copy_from racing read to smithi00140323-32 2021-01-15T19:10:32.448 INFO:tasks.rados.rados.0.smithi001.stdout:675: finishing write tid 6 to smithi00140323-22 2021-01-15T19:10:32.449 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 22 v 181 (ObjNum 240 snap 75 seq_num 240) dirty exists 2021-01-15T19:10:32.449 INFO:tasks.rados.rados.0.smithi001.stdout:675: left oid 22 (ObjNum 240 snap 75 seq_num 240) 2021-01-15T19:10:32.449 INFO:tasks.rados.rados.0.smithi001.stdout:674: finishing copy_from to smithi00140323-47 2021-01-15T19:10:32.449 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 186 (ObjNum 233 snap 72 seq_num 233) dirty exists 2021-01-15T19:10:32.450 INFO:tasks.rados.rados.0.smithi001.stdout:674: finishing copy_from racing read to smithi00140323-47 2021-01-15T19:10:32.450 INFO:tasks.rados.rados.0.smithi001.stdout:674: done (7 left) 2021-01-15T19:10:32.450 INFO:tasks.rados.rados.0.smithi001.stdout:675: done (6 left) 2021-01-15T19:10:32.450 INFO:tasks.rados.rados.0.smithi001.stdout:676: done (5 left) 2021-01-15T19:10:32.451 INFO:tasks.rados.rados.0.smithi001.stdout:677: done (4 left) 2021-01-15T19:10:32.451 INFO:tasks.rados.rados.0.smithi001.stdout:678: done (3 left) 2021-01-15T19:10:32.451 INFO:tasks.rados.rados.0.smithi001.stdout:679: done (2 left) 2021-01-15T19:10:32.452 INFO:tasks.rados.rados.0.smithi001.stdout:680: done (1 left) 2021-01-15T19:10:32.452 INFO:tasks.rados.rados.0.smithi001.stdout:681: done (0 left) 2021-01-15T19:10:32.452 INFO:tasks.rados.rados.0.smithi001.stdout:682: rollback oid 13 current snap is 75 2021-01-15T19:10:32.452 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 13 to 74 2021-01-15T19:10:32.452 INFO:tasks.rados.rados.0.smithi001.stdout:683: snap_remove snap 56 2021-01-15T19:10:32.491 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:32.490+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:32.530 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.530 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.539 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:32.592 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:32.590+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:32.663 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.663 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.672 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:32.716 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:32.715+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:32.794 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:32.793+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:32.806 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.807 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.813 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:32.897 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:32.896+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:32.979 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:32.979 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:32.989 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:33.030 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:33.029+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:33.097 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:33.096+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:33.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:33.155 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:33.164 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:33.205 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:33.204+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:33.223 INFO:tasks.rados.rados.0.smithi001.stdout:683: done (1 left) 2021-01-15T19:10:33.224 INFO:tasks.rados.rados.0.smithi001.stdout:684: rollback oid 18 current snap is 75 2021-01-15T19:10:33.224 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 18 to 69 2021-01-15T19:10:33.225 INFO:tasks.rados.rados.0.smithi001.stdout:685: read oid 36 snap 57 2021-01-15T19:10:33.225 INFO:tasks.rados.rados.0.smithi001.stdout:685: expect (ObjNum 180 snap 56 seq_num 180) 2021-01-15T19:10:33.225 INFO:tasks.rados.rados.0.smithi001.stdout:686: copy_from oid 17 from oid 33 current snap is 75 2021-01-15T19:10:33.226 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 45 current snap is 75 2021-01-15T19:10:33.227 INFO:tasks.rados.rados.0.smithi001.stdout:687: seq_num 243 ranges {0=475136,475136=638976,1114112=565248} 2021-01-15T19:10:33.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:10:33.229 INFO:tasks.rados.rados.0.smithi001.stdout:687: writing smithi00140323-45 from 0 to 475136 tid 1 2021-01-15T19:10:33.229 INFO:tasks.rados.rados.0.smithi001.stdout:687: writing smithi00140323-45 from 475136 to 1114112 tid 2 2021-01-15T19:10:33.234 INFO:tasks.rados.rados.0.smithi001.stdout:687: writing smithi00140323-45 from 1114112 to 1679360 tid 3 2021-01-15T19:10:33.234 INFO:tasks.rados.rados.0.smithi001.stdout:682: finishing rollback tid 0 to smithi00140323-13 2021-01-15T19:10:33.235 INFO:tasks.rados.rados.0.smithi001.stdout:682: finishing rollback tid 1 to smithi00140323-13 2021-01-15T19:10:33.235 INFO:tasks.rados.rados.0.smithi001.stdout:682: finishing rollback tid 2 to smithi00140323-13 2021-01-15T19:10:33.236 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 324 (ObjNum 120 snap 27 seq_num 120) dirty exists 2021-01-15T19:10:33.236 INFO:tasks.rados.rados.0.smithi001.stdout:684: finishing rollback tid 0 to smithi00140323-18 2021-01-15T19:10:33.236 INFO:tasks.rados.rados.0.smithi001.stdout:682: done (4 left) 2021-01-15T19:10:33.236 INFO:tasks.rados.rados.0.smithi001.stdout:688: copy_from oid 47 from oid 8 current snap is 75 2021-01-15T19:10:33.237 INFO:tasks.rados.rados.0.smithi001.stdout:689: read oid 49 snap 58 2021-01-15T19:10:33.237 INFO:tasks.rados.rados.0.smithi001.stdout:689: expect (ObjNum 184 snap 58 seq_num 184) 2021-01-15T19:10:33.237 INFO:tasks.rados.rados.0.smithi001.stdout:690: copy_from oid 4 from oid 41 current snap is 75 2021-01-15T19:10:33.237 INFO:tasks.rados.rados.0.smithi001.stdout:691: snap_remove snap 66 2021-01-15T19:10:33.299 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:33.298+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:33.304 INFO:teuthology.orchestra.run.smithi001.stdout:get_pg_num_history pg_num_history pg_num_history(e158 pg_nums {1={11=1,58=11,154=21},2={16=8,46=18},3={21=16,52=26}} deleted_pools ) 2021-01-15T19:10:33.304 INFO:teuthology.orchestra.run.smithi001.stdout:Importing pgid 3.11s2 2021-01-15T19:10:33.305 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8b30e322:::smithi00140323-24:50# 2021-01-15T19:10:33.311 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8b30e322:::smithi00140323-24:6e# 2021-01-15T19:10:33.316 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8b30e322:::smithi00140323-24:head# 2021-01-15T19:10:33.316 INFO:teuthology.orchestra.run.smithi001.stdout:snapset 6e=[]:{50=[4b],6e=[6b,67,66,64,63,56]} 2021-01-15T19:10:33.317 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8f2f8304:::smithi00140323-26:53# 2021-01-15T19:10:33.322 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8f2f8304:::smithi00140323-26:67# 2021-01-15T19:10:33.328 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8f2f8304:::smithi00140323-26:6d# 2021-01-15T19:10:33.334 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:8f2f8304:::smithi00140323-26:head# 2021-01-15T19:10:33.334 INFO:teuthology.orchestra.run.smithi001.stdout:snapset 6d=[]:{53=[4b],67=[67,66,64,63,56],6d=[6b]} 2021-01-15T19:10:33.335 INFO:teuthology.orchestra.run.smithi001.stdout:write_pg epoch 157 info 3.11s2( v 157'116 (0'0,157'116] local-lis/les=122/123 n=6 ec=52/21 lis/c=122/122 les/c/f=123/123/0 sis=122) 2021-01-15T19:10:33.336 INFO:teuthology.orchestra.run.smithi001.stdout:Import successful 2021-01-15T19:10:33.349 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:33.349 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:33.358 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:33.401 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:33.399+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:33.501 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:33.500+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:33.523 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:33.524 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:33.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:33.601 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:33.600+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:33.696 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:33.697 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:33.702 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:33.701+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:33.706 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:33.803 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:33.802+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:33.839 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:33.839 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:33.848 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:33.855 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /var/log/ceph/exp.3.11s2.1 2021-01-15T19:10:33.903 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:33.902+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:33.922 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-15T19:10:33.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:33.973 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:33.974 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:33.983 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:34.024 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:34.023+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:34.104 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:34.103+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:34.124 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:34.145 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:34.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:34.148 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:34.157 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:34.165 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:34.186 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:34.205 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:34.204+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:34.206 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:34.225 INFO:tasks.rados.rados.0.smithi001.stdout:691: done (7 left) 2021-01-15T19:10:34.225 INFO:tasks.rados.rados.0.smithi001.stdout:692: snap_remove snap 40 2021-01-15T19:10:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:10:34.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:34.257 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:34.260 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:34.260 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:34.293 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:34.294 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:34.294 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:34.303 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:34.304 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:10:34.304 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":166,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:10:34.216782+0000","last_up_change":"2021-01-15T19:10:27.281057+0000","last_in_change":"2021-01-15T19:09:47.269026+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"154","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-15T19:07:54.484638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":18,"pg_placement_num":18,"pg_placement_num_target":18,"pg_num_target":18,"pg_num_pending":18,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"46","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2021-01-15T19:08:08.242953+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":26,"pg_num_target":26,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"52","auid":0,"snap_mode":"selfmanaged","snap_seq":137,"snap_epoch":166,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"isaprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists","up"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":159,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":0.198638916015625,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":94,"up_from":132,"up_thru":92,"down_at":95,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":48096},{"type":"v1","addr":"172.21.15.136:6810","nonce":48096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":48096},{"type":"v1","addr":"172.21.15.136:6814","nonce":48096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":48096},{"type":"v1","addr":"172.21.15.136:6822","nonce":48096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":48096},{"type":"v1","addr":"172.21.15.136:6818","nonce":48096}]},"public_addr":"172.21.15.136:6810/48096","cluster_addr":"172.21.15.136:6814/48096","heartbeat_back_addr":"172.21.15.136:6822/48096","heartbeat_front_addr":"172.21.15.136:6818/48096","state":["exists","up"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:49.068217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:49.114373+0000","dead_epoch":159},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:50.647382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:49.296499+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:50.239164+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":65536,"last_purged_snaps_scrub":"2021-01-15T19:09:50.384906+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":65536,"last_purged_snaps_scrub":"2021-01-15T19:10:00.995180+0000","dead_epoch":95},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:50.678303+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[{"pgid":"1.d","osds":[2,4]}],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"},"isaprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"2","m":"1","name":"isaprofile","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":38,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":56,"length":1},{"begin":58,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":1},{"begin":70,"length":1},{"begin":76,"length":1},{"begin":80,"length":1},{"begin":83,"length":2},{"begin":86,"length":4},{"begin":92,"length":2},{"begin":95,"length":2},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":109,"length":2},{"begin":113,"length":1},{"begin":115,"length":1},{"begin":120,"length":2}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":120,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:10:34.307 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:34.306+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:34.318 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '154', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 18, 'pg_placement_num': 18, 'pg_placement_num_target': 18, 'pg_num_target': 18, 'pg_num_pending': 18, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '46', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '52', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 137, 'snap_epoch': 166, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}] 2021-01-15T19:10:34.318 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op apply-layout-settings --pool device_health_metrics 2021-01-15T19:10:34.408 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:34.407+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:34.439 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:34.439 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:34.440 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:34.440 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:34.451 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:34.509 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:34.508+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:34.582 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:34.582 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:34.583 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:34.583 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:34.583 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:34.592 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:34.636 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:34.635+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:34.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2021-01-15T19:10:34.710 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:34.709+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:34.728 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:34.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 101.47181786100001, 2021-01-15T19:10:34.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:34.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:34.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:34.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:34.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:34.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 101.471652748, 2021-01-15T19:10:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:34.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:34.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:34.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:34.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:34.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:34.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:34.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:34.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:34.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:34.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:34.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:34.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:34.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:34.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:34.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:34.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:34.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:34.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:34.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:34.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:34.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:34.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:34.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:34.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:34.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:34.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:34.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.764 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.765 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:723 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 67] snapc 2d=[2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e71)", 2021-01-15T19:10:34.765 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:34.765 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 95.464372698999995, 2021-01-15T19:10:34.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001, 2021-01-15T19:10:34.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.766 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 723 2021-01-15T19:10:34.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.767 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:34.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260707+0000", 2021-01-15T19:10:34.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7102000000000003e-05 2021-01-15T19:10:34.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260764+0000", 2021-01-15T19:10:34.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.9449999999999993e-06 2021-01-15T19:10:34.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260772+0000", 2021-01-15T19:10:34.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2884999999999999e-05 2021-01-15T19:10:34.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260785+0000", 2021-01-15T19:10:34.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9159999999999998e-06 2021-01-15T19:10:34.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260791+0000", 2021-01-15T19:10:34.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0031319619999999999 2021-01-15T19:10:34.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263923+0000", 2021-01-15T19:10:34.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7731000000000006e-05 2021-01-15T19:10:34.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.263990+0000", 2021-01-15T19:10:34.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0094480269999999995 2021-01-15T19:10:34.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.273438+0000", 2021-01-15T19:10:34.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0072993040000000004 2021-01-15T19:10:34.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280738+0000", 2021-01-15T19:10:34.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030709750000000001 2021-01-15T19:10:34.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283809+0000", 2021-01-15T19:10:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013436899999999999 2021-01-15T19:10:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.283943+0000", 2021-01-15T19:10:34.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023236216000000001 2021-01-15T19:10:34.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.787 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:34.787 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:34.787 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 95.464246066000001, 2021-01-15T19:10:34.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:34.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:34.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.789 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:34.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.791 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:34.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:34.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:34.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:34.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:34.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:34.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:34.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:34.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:34.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:34.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:34.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:34.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.381509101000006, 2021-01-15T19:10:34.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:34.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:34.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:34.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:34.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:34.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:34.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:34.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:34.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.381359486999997, 2021-01-15T19:10:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:34.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:34.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:34.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:34.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:34.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:34.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:34.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:34.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:34.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:34.858 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.361061805000006, 2021-01-15T19:10:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:34.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:34.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:34.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:34.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:34.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:34.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.878 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.879 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.358005153999997, 2021-01-15T19:10:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:34.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:34.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:34.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:34.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:34.898 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.354991892000001, 2021-01-15T19:10:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.900 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.901 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:34.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:34.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:34.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:34.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:34.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:34.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:34.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:34.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.352244280999997, 2021-01-15T19:10:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:34.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:34.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:34.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:34.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:34.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:34.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.939 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.351810459000006, 2021-01-15T19:10:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:34.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:34.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:34.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:34.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:34.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:34.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:34.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:34.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:34.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:34.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.959 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.351686299999997, 2021-01-15T19:10:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:34.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:34.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.961 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:34.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:34.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:34.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:34.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:34.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:34.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:34.978 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 82.247009765000001, 2021-01-15T19:10:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:34.979 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.980 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:34.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:34.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:34.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:34.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:34.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:34.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:34.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:34.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:34.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:34.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:35.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(osd.0.13:8 3.cs0 3:30718dd9:::smithi00140323-34:head [assert-version v110,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e85)", 2021-01-15T19:10:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.366788686000007, 2021-01-15T19:10:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001, 2021-01-15T19:10:35.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358291+0000", 2021-01-15T19:10:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0516000000000002e-05 2021-01-15T19:10:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358341+0000", 2021-01-15T19:10:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.49e-06 2021-01-15T19:10:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358350+0000", 2021-01-15T19:10:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4089e-05 2021-01-15T19:10:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358364+0000", 2021-01-15T19:10:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4339999999999997e-06 2021-01-15T19:10:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.358370+0000", 2021-01-15T19:10:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0054197610000000004 2021-01-15T19:10:35.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.363790+0000", 2021-01-15T19:10:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000288485 2021-01-15T19:10:35.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364079+0000", 2021-01-15T19:10:35.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017367614999999999 2021-01-15T19:10:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381446+0000", 2021-01-15T19:10:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023155390000000001 2021-01-15T19:10:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:35.014 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.337070529000002, 2021-01-15T19:10:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:35.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:35.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:35.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:35.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:35.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:35.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=259, last_complete=85'127, committed=1, applied=1))", 2021-01-15T19:10:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.332984362000005, 2021-01-15T19:10:35.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001, 2021-01-15T19:10:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392095+0000", 2021-01-15T19:10:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2506999999999997e-05 2021-01-15T19:10:35.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392148+0000", 2021-01-15T19:10:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0441e-05 2021-01-15T19:10:35.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392158+0000", 2021-01-15T19:10:35.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5922000000000001e-05 2021-01-15T19:10:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392174+0000", 2021-01-15T19:10:35.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.418e-06 2021-01-15T19:10:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392182+0000", 2021-01-15T19:10:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022620487000000002 2021-01-15T19:10:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414802+0000", 2021-01-15T19:10:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060367899999999998 2021-01-15T19:10:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415406+0000", 2021-01-15T19:10:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023310454000000001 2021-01-15T19:10:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=260, last_complete=85'128, committed=1, applied=1))", 2021-01-15T19:10:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:35.038 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.332848611000003, 2021-01-15T19:10:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506, 2021-01-15T19:10:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392231+0000", 2021-01-15T19:10:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0380000000000002e-05 2021-01-15T19:10:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392281+0000", 2021-01-15T19:10:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0660000000000004e-06 2021-01-15T19:10:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392289+0000", 2021-01-15T19:10:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3563e-05 2021-01-15T19:10:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392303+0000", 2021-01-15T19:10:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.967e-06 2021-01-15T19:10:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.392308+0000", 2021-01-15T19:10:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023215962999999999 2021-01-15T19:10:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415524+0000", 2021-01-15T19:10:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00041456700000000002 2021-01-15T19:10:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415939+0000", 2021-01-15T19:10:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023707506 2021-01-15T19:10:35.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:35.050 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.494349628000002, 2021-01-15T19:10:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.051 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:35.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:35.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:35.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:35.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:35.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:35.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:35.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:35.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.519770164000001, 2021-01-15T19:10:35.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.067 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.068 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:35.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:35.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:35.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:35.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:35.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:35.086 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 24.519638502999999, 2021-01-15T19:10:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.087 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.088 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:35.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:35.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:35.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:35.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:35.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:35.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:35.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:35.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:35.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.110 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.110 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:35.111 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:35.112 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:35.110+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:35.212 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:35.211+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:35.228 INFO:tasks.rados.rados.0.smithi001.stdout:692: done (7 left) 2021-01-15T19:10:35.229 INFO:tasks.rados.rados.0.smithi001.stdout:693: rmattr oid 3 current snap is 75 2021-01-15T19:10:35.229 INFO:tasks.rados.rados.0.smithi001.stdout:693: done (7 left) 2021-01-15T19:10:35.230 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 39 current snap is 75 2021-01-15T19:10:35.230 INFO:tasks.rados.rados.0.smithi001.stdout:684: finishing rollback tid 1 to smithi00140323-18 2021-01-15T19:10:35.231 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 18 v 233 (ObjNum 0 snap 0 seq_num 0) dirty dne 2021-01-15T19:10:35.231 INFO:tasks.rados.rados.0.smithi001.stdout:687: finishing write tid 1 to smithi00140323-45 2021-01-15T19:10:35.231 INFO:tasks.rados.rados.0.smithi001.stdout:687: finishing write tid 2 to smithi00140323-45 2021-01-15T19:10:35.232 INFO:tasks.rados.rados.0.smithi001.stdout:687: finishing write tid 3 to smithi00140323-45 2021-01-15T19:10:35.232 INFO:tasks.rados.rados.0.smithi001.stdout:687: finishing write tid 4 to smithi00140323-45 2021-01-15T19:10:35.233 INFO:tasks.rados.rados.0.smithi001.stdout:687: finishing write tid 5 to smithi00140323-45 2021-01-15T19:10:35.233 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 309 (ObjNum 243 snap 75 seq_num 243) dirty exists 2021-01-15T19:10:35.233 INFO:tasks.rados.rados.0.smithi001.stdout:687: left oid 45 (ObjNum 243 snap 75 seq_num 243) 2021-01-15T19:10:35.234 INFO:tasks.rados.rados.0.smithi001.stdout:685: expect (ObjNum 180 snap 56 seq_num 180) 2021-01-15T19:10:35.300 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:35.300 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:35.301 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:35.301 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:35.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:35.314 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:35.312+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:35.399 INFO:tasks.rados.rados.0.smithi001.stdout:686: finishing copy_from to smithi00140323-17 2021-01-15T19:10:35.399 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 17 v 265 (ObjNum 229 snap 71 seq_num 229) dirty exists 2021-01-15T19:10:35.400 INFO:tasks.rados.rados.0.smithi001.stdout:686: finishing copy_from racing read to smithi00140323-17 2021-01-15T19:10:35.400 INFO:tasks.rados.rados.0.smithi001.stdout:689: expect (ObjNum 184 snap 58 seq_num 184) 2021-01-15T19:10:35.415 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:35.413+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:35.442 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:35.443 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:35.443 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:35.443 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:35.444 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:35.453 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:35.503 INFO:tasks.rados.rados.0.smithi001.stdout:690: finishing copy_from to smithi00140323-4 2021-01-15T19:10:35.504 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 4 v 135 (ObjNum 241 snap 75 seq_num 241) dirty exists 2021-01-15T19:10:35.504 INFO:tasks.rados.rados.0.smithi001.stdout:690: finishing copy_from racing read to smithi00140323-4 2021-01-15T19:10:35.504 INFO:tasks.rados.rados.0.smithi001.stdout:688: finishing copy_from to smithi00140323-47 2021-01-15T19:10:35.505 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 186 (ObjNum 217 snap 68 seq_num 217) dirty exists 2021-01-15T19:10:35.505 INFO:tasks.rados.rados.0.smithi001.stdout:688: finishing copy_from racing read to smithi00140323-47 2021-01-15T19:10:35.506 INFO:tasks.rados.rados.0.smithi001.stdout:694: seq_num 244 ranges {0=491520,491520=573440,1064960=458752,1523712=172032} 2021-01-15T19:10:35.506 INFO:tasks.rados.rados.0.smithi001.stdout:694: writing smithi00140323-39 from 0 to 491520 tid 1 2021-01-15T19:10:35.509 INFO:tasks.rados.rados.0.smithi001.stdout:694: writing smithi00140323-39 from 491520 to 1064960 tid 2 2021-01-15T19:10:35.511 INFO:tasks.rados.rados.0.smithi001.stdout:694: writing smithi00140323-39 from 1064960 to 1523712 tid 3 2021-01-15T19:10:35.514 INFO:tasks.rados.rados.0.smithi001.stdout:694: writing smithi00140323-39 from 1523712 to 1695744 tid 4 2021-01-15T19:10:35.514 INFO:tasks.rados.rados.0.smithi001.stdout:694: finishing write tid 1 to smithi00140323-39 2021-01-15T19:10:35.515 INFO:tasks.rados.rados.0.smithi001.stdout:684: done (7 left) 2021-01-15T19:10:35.515 INFO:tasks.rados.rados.0.smithi001.stdout:685: done (6 left) 2021-01-15T19:10:35.515 INFO:tasks.rados.rados.0.smithi001.stdout:686: done (5 left) 2021-01-15T19:10:35.516 INFO:tasks.rados.rados.0.smithi001.stdout:687: done (4 left) 2021-01-15T19:10:35.516 INFO:tasks.rados.rados.0.smithi001.stdout:688: done (3 left) 2021-01-15T19:10:35.517 INFO:tasks.rados.rados.0.smithi001.stdout:689: done (2 left) 2021-01-15T19:10:35.517 INFO:tasks.rados.rados.0.smithi001.stdout:690: done (1 left) 2021-01-15T19:10:35.517 INFO:tasks.rados.rados.0.smithi001.stdout:695: read oid 7 snap -1 2021-01-15T19:10:35.518 INFO:tasks.rados.rados.0.smithi001.stdout:695: expect (ObjNum 183 snap 58 seq_num 183) 2021-01-15T19:10:35.518 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 33 current snap is 75 2021-01-15T19:10:35.519 INFO:tasks.rados.rados.0.smithi001.stdout:696: seq_num 245 ranges {1441792=507904,1949696=458752,2408448=671744,3080192=73728} 2021-01-15T19:10:35.522 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:35.521+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:35.524 INFO:tasks.rados.rados.0.smithi001.stdout:696: writing smithi00140323-33 from 1441792 to 1949696 tid 1 2021-01-15T19:10:35.527 INFO:tasks.rados.rados.0.smithi001.stdout:696: writing smithi00140323-33 from 1949696 to 2408448 tid 2 2021-01-15T19:10:35.531 INFO:tasks.rados.rados.0.smithi001.stdout:696: writing smithi00140323-33 from 2408448 to 3080192 tid 3 2021-01-15T19:10:35.532 INFO:tasks.rados.rados.0.smithi001.stdout:696: writing smithi00140323-33 from 3080192 to 3153920 tid 4 2021-01-15T19:10:35.533 INFO:tasks.rados.rados.0.smithi001.stdout:697: snap_remove snap 73 2021-01-15T19:10:35.533 INFO:tasks.rados.rados.0.smithi001.stdout:694: finishing write tid 2 to smithi00140323-39 2021-01-15T19:10:35.533 INFO:tasks.rados.rados.0.smithi001.stdout:694: finishing write tid 3 to smithi00140323-39 2021-01-15T19:10:35.533 INFO:tasks.rados.rados.0.smithi001.stdout:694: finishing write tid 4 to smithi00140323-39 2021-01-15T19:10:35.534 INFO:tasks.rados.rados.0.smithi001.stdout:694: finishing write tid 5 to smithi00140323-39 2021-01-15T19:10:35.534 INFO:tasks.rados.rados.0.smithi001.stdout:694: finishing write tid 6 to smithi00140323-39 2021-01-15T19:10:35.534 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 39 v 88 (ObjNum 244 snap 75 seq_num 244) dirty exists 2021-01-15T19:10:35.534 INFO:tasks.rados.rados.0.smithi001.stdout:694: left oid 39 (ObjNum 244 snap 75 seq_num 244) 2021-01-15T19:10:35.541 INFO:teuthology.orchestra.run.smithi001.stderr:Nothing to do for non-filestore backend 2021-01-15T19:10:35.623 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:35.622+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:35.643 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:35.643 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:35.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:35.643 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:35.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:35.644 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:35.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 112.205163058, 2021-01-15T19:10:35.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:10:35.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:10:35.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.646 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:35.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:35.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:10:35.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:10:35.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:10:35.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:10:35.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:10:35.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:10:35.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:35.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:10:35.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:10:35.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:10:35.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:10:35.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:10:35.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:10:35.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:10:35.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:10:35.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:10:35.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:10:35.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:10:35.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:10:35.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:10:35.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:10:35.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:10:35.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:10:35.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:10:35.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:10:35.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:10:35.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:10:35.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:10:35.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:10:35.666 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.666 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.666 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.667 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:35.667 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:35.667 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 112.205023758, 2021-01-15T19:10:35.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:10:35.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.669 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:10:35.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.669 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:35.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:35.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:10:35.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:10:35.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:35.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:10:35.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:10:35.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:10:35.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:10:35.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:10:35.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:10:35.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:10:35.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:10:35.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:10:35.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:10:35.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:10:35.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:10:35.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:10:35.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:10:35.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:10:35.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:10:35.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:10:35.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:10:35.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:10:35.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:10:35.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:10:35.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:10:35.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:10:35.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:10:35.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:10:35.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:10:35.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.687 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.687 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.688 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:35.688 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:35.688 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 112.204902652, 2021-01-15T19:10:35.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:10:35.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.689 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.690 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:10:35.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.691 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:35.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:35.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:10:35.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:10:35.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:10:35.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:10:35.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:10:35.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:10:35.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:10:35.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:10:35.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:10:35.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:10:35.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:10:35.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:10:35.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:10:35.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:10:35.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:10:35.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:10:35.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:10:35.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:10:35.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:10:35.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:10:35.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:10:35.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:10:35.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:10:35.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:10:35.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:10:35.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:10:35.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:10:35.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:10:35.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:10:35.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.710 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:35.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:35.710 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.43462177799999, 2021-01-15T19:10:35.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:10:35.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:10:35.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.712 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:35.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:35.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:10:35.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:10:35.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:10:35.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:10:35.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:10:35.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:10:35.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:10:35.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:10:35.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:10:35.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:10:35.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:10:35.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:10:35.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:10:35.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:10:35.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:10:35.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:10:35.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:10:35.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:10:35.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:10:35.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:10:35.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:10:35.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:10:35.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:10:35.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:10:35.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:10:35.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.728 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.729 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:35.729 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:35.730 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.4256436, 2021-01-15T19:10:35.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:10:35.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.731 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:10:35.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.732 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:35.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:35.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:10:35.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:10:35.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:10:35.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:10:35.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:10:35.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:10:35.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:35.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:10:35.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:10:35.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:35.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:10:35.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:10:35.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:10:35.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:10:35.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:10:35.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:10:35.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:10:35.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:10:35.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:10:35.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:10:35.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:10:35.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:10:35.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:10:35.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:10:35.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:10:35.748 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.748 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.749 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:35.750 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:35.750 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.4251239, 2021-01-15T19:10:35.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:10:35.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.751 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:10:35.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.752 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:35.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:35.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:10:35.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:10:35.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:10:35.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:10:35.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:10:35.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:10:35.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:10:35.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:10:35.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:10:35.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:10:35.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:10:35.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:10:35.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:10:35.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:10:35.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:10:35.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:10:35.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:10:35.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:10:35.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:10:35.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:10:35.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:10:35.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:10:35.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:10:35.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:10:35.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:10:35.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.768 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.769 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:35.769 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.42344294599999, 2021-01-15T19:10:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:10:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:10:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:35.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:35.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:10:35.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:10:35.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:10:35.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:10:35.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:10:35.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:10:35.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:10:35.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:10:35.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:10:35.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:35.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:10:35.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:10:35.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:10:35.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:10:35.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:10:35.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:10:35.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:10:35.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:10:35.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:10:35.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:10:35.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:10:35.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:10:35.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:10:35.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:10:35.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:10:35.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.789 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:35.789 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:35.789 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.423309992, 2021-01-15T19:10:35.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:10:35.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.790 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:10:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.792 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:35.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:35.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:10:35.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:10:35.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:10:35.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:10:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:10:35.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:10:35.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:10:35.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:10:35.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:10:35.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:10:35.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:35.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:10:35.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:10:35.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:10:35.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:10:35.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:35.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:10:35.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:10:35.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:10:35.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:10:35.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:10:35.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:10:35.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:10:35.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:10:35.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.807 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.808 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:35.808 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:35.808 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.460778722, 2021-01-15T19:10:35.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:10:35.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.809 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.810 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:10:35.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.810 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:35.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:35.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:10:35.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:10:35.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:10:35.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:10:35.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:10:35.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:10:35.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:10:35.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:10:35.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:10:35.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:10:35.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:10:35.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:35.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:10:35.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:10:35.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:10:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:10:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:10:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:10:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:10:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:10:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:10:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:10:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:10:35.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:10:35.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:10:35.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:10:35.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.828 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:35.828 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:35.828 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.460612129, 2021-01-15T19:10:35.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:10:35.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.829 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.830 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:10:35.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.830 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:35.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:35.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:10:35.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:10:35.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:10:35.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:10:35.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:10:35.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:10:35.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:10:35.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:10:35.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:10:35.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:10:35.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:10:35.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:35.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:10:35.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:10:35.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:10:35.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:10:35.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:35.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:10:35.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:10:35.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:10:35.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:10:35.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:35.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:10:35.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:10:35.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:10:35.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:10:35.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:10:35.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:10:35.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:10:35.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:10:35.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.848 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.849 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:35.849 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:35.849 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 83.262073334999997, 2021-01-15T19:10:35.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:10:35.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.850 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.851 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:10:35.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.852 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:35.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:35.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:10:35.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:10:35.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:10:35.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:10:35.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:10:35.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:10:35.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:10:35.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:10:35.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:10:35.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:10:35.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:10:35.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:10:35.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:10:35.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:10:35.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:10:35.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:10:35.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:10:35.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:10:35.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:10:35.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:10:35.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:10:35.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:10:35.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:10:35.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:10:35.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:10:35.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.868 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.869 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:35.869 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:35.870 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 83.259803289999994, 2021-01-15T19:10:35.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:10:35.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.871 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:10:35.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.872 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:35.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:35.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:10:35.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:10:35.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:10:35.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:10:35.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:10:35.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:10:35.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:10:35.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:10:35.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:10:35.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:10:35.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:10:35.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:10:35.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:10:35.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:10:35.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:10:35.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:10:35.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:10:35.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:10:35.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:10:35.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:10:35.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:10:35.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:10:35.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:10:35.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:10:35.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:10:35.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.888 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:35.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:35.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 83.255265098999999, 2021-01-15T19:10:35.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:10:35.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:10:35.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.891 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:35.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:35.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:10:35.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:10:35.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:10:35.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:10:35.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:10:35.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:10:35.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:35.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:10:35.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:10:35.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:10:35.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:10:35.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:10:35.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:10:35.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:10:35.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:10:35.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:10:35.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:10:35.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:10:35.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:10:35.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:10:35.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:10:35.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:10:35.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:10:35.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:10:35.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:10:35.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.907 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.908 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:35.909 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:35.909 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 83.252679227000002, 2021-01-15T19:10:35.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:10:35.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:10:35.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.911 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:35.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:35.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:10:35.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:10:35.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:10:35.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:10:35.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:10:35.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:10:35.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:10:35.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:10:35.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:10:35.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:10:35.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:10:35.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:10:35.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:10:35.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:10:35.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:10:35.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:10:35.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:10:35.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:10:35.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:10:35.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:10:35.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:10:35.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:10:35.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:10:35.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:10:35.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:10:35.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.928 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.928 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.929 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:35.929 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:35.929 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 83.252304891999998, 2021-01-15T19:10:35.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:10:35.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.930 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.930 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.931 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:10:35.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.931 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:35.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:35.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:10:35.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:10:35.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:10:35.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:10:35.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:10:35.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:10:35.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:10:35.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:10:35.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:10:35.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:10:35.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:10:35.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:10:35.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:10:35.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:10:35.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:10:35.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:10:35.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:10:35.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:10:35.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:10:35.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:10:35.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:10:35.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:10:35.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:10:35.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:10:35.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:10:35.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.948 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:35.949 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:35.949 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 83.252168397000005, 2021-01-15T19:10:35.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:10:35.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:10:35.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.951 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:35.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:35.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:10:35.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:10:35.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:10:35.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:10:35.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:10:35.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:10:35.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:10:35.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:10:35.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:10:35.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:10:35.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:10:35.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:35.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:10:35.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:10:35.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:10:35.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:10:35.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:35.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:10:35.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:10:35.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:10:35.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:10:35.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:10:35.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:10:35.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:10:35.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:10:35.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.966 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.966 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:10:35.967 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:35.967 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 81.269770676999997, 2021-01-15T19:10:35.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:10:35.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:35.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.968 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:10:35.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.969 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:35.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:35.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:10:35.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:10:35.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:10:35.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:10:35.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:10:35.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:10:35.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:10:35.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:10:35.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:10:35.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:10:35.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:10:35.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:35.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:10:35.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:10:35.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:10:35.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:10:35.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:35.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:10:35.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:10:35.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:10:35.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:10:35.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:10:35.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:10:35.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:10:35.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:10:35.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:10:35.984 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:35.984 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.123017619000001, 2021-01-15T19:10:35.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:10:35.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:35.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:35.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:35.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:35.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:35.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:10:35.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.986 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:35.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:35.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:35.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:35.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:35.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:35.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:10:35.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:35.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:10:35.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:10:35.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:35.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:10:35.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:10:35.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:35.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:10:35.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:10:35.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:35.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:10:35.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:10:35.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:35.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:10:35.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:10:35.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:35.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:10:35.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:10:35.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:35.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:10:35.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:10:35.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:35.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:10:35.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:10:35.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:35.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:10:35.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:10:35.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:35.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:10:35.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:10:35.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.998 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:35.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:35.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:35.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:35.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:10:35.999 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:36.000 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 14.122854687, 2021-01-15T19:10:36.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:10:36.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:36.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:36.000 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:36.001 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:36.001 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:36.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:10:36.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.001 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:36.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:36.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:36.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:36.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:36.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:36.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:10:36.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:36.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:10:36.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:10:36.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:36.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:10:36.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:10:36.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:36.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:10:36.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:10:36.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:36.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:10:36.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:10:36.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:36.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:10:36.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:10:36.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:36.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:10:36.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:10:36.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:36.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:10:36.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:10:36.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:36.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:10:36.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:10:36.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:36.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:10:36.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:10:36.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:36.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:10:36.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:10:36.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:36.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:10:36.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:10:36.018 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:36.018 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:36.019 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:36.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.019 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:10:36.020 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:36.020 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.3491820499999996, 2021-01-15T19:10:36.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:10:36.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:36.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:36.021 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:36.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:36.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:36.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:36.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:36.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:36.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:10:36.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:36.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:10:36.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:10:36.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:36.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:10:36.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:10:36.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:36.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:10:36.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:10:36.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:36.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:10:36.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:10:36.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:36.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:10:36.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:10:36.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:36.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:10:36.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:10:36.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:36.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:10:36.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:10:36.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:36.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:10:36.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:10:36.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:36.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:36.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:36.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:10:36.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:10:36.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:36.032 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:36.032 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:36.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:36.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:36.033 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:36.034 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:36.033+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:36.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:36.062 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op apply-layout-settings --pool rbd 2021-01-15T19:10:36.146 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:36.145+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:36.232 INFO:tasks.rados.rados.0.smithi001.stdout:694: done (3 left) 2021-01-15T19:10:36.233 INFO:tasks.rados.rados.0.smithi001.stdout:697: done (2 left) 2021-01-15T19:10:36.233 INFO:tasks.rados.rados.0.smithi001.stdout:698: setattr oid 11 current snap is 75 2021-01-15T19:10:36.234 INFO:tasks.rados.rados.0.smithi001.stdout:696: finishing write tid 1 to smithi00140323-33 2021-01-15T19:10:36.234 INFO:tasks.rados.rados.0.smithi001.stdout:695: expect (ObjNum 183 snap 58 seq_num 183) 2021-01-15T19:10:36.240 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:36.235+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:36.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:10:36.246 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:36.246 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:36.247 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:36.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:36.262 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:36.337 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:36.336+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:36.438 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:36.437+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:36.440 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:36.440 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:36.441 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:36.441 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:36.456 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:36.488 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:36.529 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:36.539 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:36.538+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:36.610 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:36.641 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:36.640+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:36.646 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:36.661 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:36.662 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:36.670 INFO:tasks.rados.rados.0.smithi001.stdout:696: finishing write tid 2 to smithi00140323-33 2021-01-15T19:10:36.671 INFO:tasks.rados.rados.0.smithi001.stdout:696: finishing write tid 3 to smithi00140323-33 2021-01-15T19:10:36.671 INFO:tasks.rados.rados.0.smithi001.stdout:696: finishing write tid 4 to smithi00140323-33 2021-01-15T19:10:36.671 INFO:tasks.rados.rados.0.smithi001.stdout:696: finishing write tid 5 to smithi00140323-33 2021-01-15T19:10:36.671 INFO:tasks.rados.rados.0.smithi001.stdout:696: finishing write tid 6 to smithi00140323-33 2021-01-15T19:10:36.671 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 333 (ObjNum 245 snap 75 seq_num 245) dirty exists 2021-01-15T19:10:36.672 INFO:tasks.rados.rados.0.smithi001.stdout:696: left oid 33 (ObjNum 245 snap 75 seq_num 245) 2021-01-15T19:10:36.672 INFO:tasks.rados.rados.0.smithi001.stdout:695: done (2 left) 2021-01-15T19:10:36.672 INFO:tasks.rados.rados.0.smithi001.stdout:696: done (1 left) 2021-01-15T19:10:36.672 INFO:tasks.rados.rados.0.smithi001.stdout:699: snap_create 2021-01-15T19:10:36.674 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:36.675 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 340 (ObjNum 1 snap 32581 seq_num 179728032) dirty exists 2021-01-15T19:10:36.683 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:36.710 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:36.710 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.710 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:36.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:36.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.711 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.712 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.712 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.712 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:36.713 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:36.713 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:36.754 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:36.753+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:36.814 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:36.815 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:36.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:36.869 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:36.868+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:36.944 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:36.943+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:36.949 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:36.949 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:36.959 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:37.044 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:37.043+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:37.124 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:37.124 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:37.133 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:37.174 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:37.173+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:37.237 INFO:tasks.rados.rados.0.smithi001.stdout:698: done (1 left) 2021-01-15T19:10:37.237 INFO:tasks.rados.rados.0.smithi001.stdout:699: done (0 left) 2021-01-15T19:10:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:700: rollback oid 33 current snap is 76 2021-01-15T19:10:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 33 to 75 2021-01-15T19:10:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:701: read oid 13 snap 58 2021-01-15T19:10:37.238 INFO:tasks.rados.rados.0.smithi001.stdout:701: expect (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:10:37.239 INFO:tasks.rados.rados.0.smithi001.stdout:702: snap_create 2021-01-15T19:10:37.245 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:10:37.246 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:37.245+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:37.251 INFO:tasks.rados.rados.0.smithi001.stdout:700: finishing rollback tid 0 to smithi00140323-33 2021-01-15T19:10:37.252 INFO:tasks.rados.rados.0.smithi001.stdout:700: finishing rollback tid 1 to smithi00140323-33 2021-01-15T19:10:37.252 INFO:tasks.rados.rados.0.smithi001.stdout:700: finishing rollback tid 2 to smithi00140323-33 2021-01-15T19:10:37.253 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 33 v 348 (ObjNum 245 snap 75 seq_num 245) dirty exists 2021-01-15T19:10:37.278 INFO:tasks.rados.rados.0.smithi001.stdout:701: expect (ObjNum 165 snap 44 seq_num 165) 2021-01-15T19:10:37.311 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:37.312 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:37.321 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:37.347 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:37.345+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:37.447 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:37.446+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:Nothing to do for non-filestore backend 2021-01-15T19:10:37.486 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:37.486 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:37.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:37.580 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:37.579+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:37.650 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:37.648+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:37.661 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:37.661 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:37.672 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:37.751 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:37.750+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:37.836 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:37.836 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:37.845 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:37.890 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:37.889+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:37.952 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:37.951+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:37.971 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:37.971 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:37.980 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:37.991 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op apply-layout-settings --pool unique_pool_0 2021-01-15T19:10:38.075 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:38.074+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:38.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:38.145 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:38.154 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:38.153+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:38.155 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:38.203 INFO:tasks.rados.rados.0.smithi001.stdout:700: done (2 left) 2021-01-15T19:10:38.204 INFO:tasks.rados.rados.0.smithi001.stdout:701: done (1 left) 2021-01-15T19:10:38.204 INFO:tasks.rados.rados.0.smithi001.stdout:702: done (0 left) 2021-01-15T19:10:38.204 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 2 current snap is 77 2021-01-15T19:10:38.204 INFO:tasks.rados.rados.0.smithi001.stdout:703: seq_num 246 ranges {0=524288,524288=745472,1269760=425984,1695744=32768} 2021-01-15T19:10:38.207 INFO:tasks.rados.rados.0.smithi001.stdout:703: writing smithi00140323-2 from 0 to 524288 tid 1 2021-01-15T19:10:38.211 INFO:tasks.rados.rados.0.smithi001.stdout:703: writing smithi00140323-2 from 524288 to 1269760 tid 2 2021-01-15T19:10:38.214 INFO:tasks.rados.rados.0.smithi001.stdout:703: writing smithi00140323-2 from 1269760 to 1695744 tid 3 2021-01-15T19:10:38.216 INFO:tasks.rados.rados.0.smithi001.stdout:703: writing smithi00140323-2 from 1695744 to 1728512 tid 4 2021-01-15T19:10:38.216 INFO:tasks.rados.rados.0.smithi001.stdout:704: rmattr oid 45 current snap is 77 2021-01-15T19:10:38.216 INFO:tasks.rados.rados.0.smithi001.stdout:704: done (1 left) 2021-01-15T19:10:38.216 INFO:tasks.rados.rados.0.smithi001.stdout:705: copy_from oid 14 from oid 25 current snap is 77 2021-01-15T19:10:38.216 INFO:tasks.rados.rados.0.smithi001.stdout:706: read oid 17 snap 68 2021-01-15T19:10:38.217 INFO:tasks.rados.rados.0.smithi001.stdout:706: expect (ObjNum 203 snap 62 seq_num 203) 2021-01-15T19:10:38.217 INFO:tasks.rados.rados.0.smithi001.stdout:707: read oid 28 snap -1 2021-01-15T19:10:38.217 INFO:tasks.rados.rados.0.smithi001.stdout:707: expect (ObjNum 167 snap 47 seq_num 167) 2021-01-15T19:10:38.217 INFO:tasks.rados.rados.0.smithi001.stdout:708: rollback oid 45 current snap is 77 2021-01-15T19:10:38.218 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 45 to 57 2021-01-15T19:10:38.218 INFO:tasks.rados.rados.0.smithi001.stdout:709: copy_from oid 50 from oid 36 current snap is 77 2021-01-15T19:10:38.218 INFO:tasks.rados.rados.0.smithi001.stdout:710: read oid 1 snap -1 2021-01-15T19:10:38.218 INFO:tasks.rados.rados.0.smithi001.stdout:710: expect (ObjNum 0 snap 0 seq_num 0) 2021-01-15T19:10:38.218 INFO:tasks.rados.rados.0.smithi001.stdout:711: snap_create 2021-01-15T19:10:38.219 INFO:tasks.rados.rados.0.smithi001.stdout:710: expect (ObjNum 0 snap 0 seq_num 0) 2021-01-15T19:10:38.224 INFO:tasks.rados.rados.0.smithi001.stdout:703: finishing write tid 1 to smithi00140323-2 2021-01-15T19:10:38.224 INFO:tasks.rados.rados.0.smithi001.stdout:703: finishing write tid 2 to smithi00140323-2 2021-01-15T19:10:38.225 INFO:tasks.rados.rados.0.smithi001.stdout:703: finishing write tid 3 to smithi00140323-2 2021-01-15T19:10:38.226 INFO:tasks.rados.rados.0.smithi001.stdout:703: finishing write tid 4 to smithi00140323-2 2021-01-15T19:10:38.228 INFO:tasks.rados.rados.0.smithi001.stdout:703: finishing write tid 5 to smithi00140323-2 2021-01-15T19:10:38.228 INFO:tasks.rados.rados.0.smithi001.stdout:708: finishing rollback tid 0 to smithi00140323-45 2021-01-15T19:10:38.228 INFO:tasks.rados.rados.0.smithi001.stdout:708: finishing rollback tid 1 to smithi00140323-45 2021-01-15T19:10:38.231 INFO:tasks.rados.rados.0.smithi001.stdout:706: expect (ObjNum 203 snap 62 seq_num 203) 2021-01-15T19:10:38.255 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:38.254+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:38.288 INFO:tasks.rados.rados.0.smithi001.stdout:708: finishing rollback tid 2 to smithi00140323-45 2021-01-15T19:10:38.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 352 (ObjNum 89 snap 16 seq_num 89) dirty exists 2021-01-15T19:10:38.288 INFO:tasks.rados.rados.0.smithi001.stdout:703: finishing write tid 6 to smithi00140323-2 2021-01-15T19:10:38.288 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 2 v 187 (ObjNum 246 snap 77 seq_num 246) dirty exists 2021-01-15T19:10:38.289 INFO:tasks.rados.rados.0.smithi001.stdout:703: left oid 2 (ObjNum 246 snap 77 seq_num 246) 2021-01-15T19:10:38.289 INFO:tasks.rados.rados.0.smithi001.stdout:707: expect (ObjNum 167 snap 47 seq_num 167) 2021-01-15T19:10:38.336 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:38.337 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:38.346 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:38.390 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:38.389+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:38.456 INFO:tasks.rados.rados.0.smithi001.stdout:705: finishing copy_from to smithi00140323-14 2021-01-15T19:10:38.456 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 14 v 277 (ObjNum 214 snap 68 seq_num 214) dirty exists 2021-01-15T19:10:38.456 INFO:tasks.rados.rados.0.smithi001.stdout:705: finishing copy_from racing read to smithi00140323-14 2021-01-15T19:10:38.456 INFO:tasks.rados.rados.0.smithi001.stdout:709: finishing copy_from to smithi00140323-50 2021-01-15T19:10:38.457 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 186 (ObjNum 233 snap 72 seq_num 233) dirty exists 2021-01-15T19:10:38.457 INFO:tasks.rados.rados.0.smithi001.stdout:709: finishing copy_from racing read to smithi00140323-50 2021-01-15T19:10:38.457 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:38.455+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:38.471 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:38.471 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:38.481 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:38.565 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:38.564+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:38.646 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:38.647 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:38.656 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:38.657 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:38.657+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:38.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:38.758 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:38.757+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:38.790 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:38.790 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:38.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:38.859 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:38.858+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:38.880 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:38.903 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:38.923 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:38.944 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:38.947 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:38.947 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:38.957 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:38.965 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:38.966 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:38.964+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:38.985 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:39.007 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:39.007 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.007 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:39.007 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:39.008 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.008 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.008 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.009 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.009 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.009 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:39.010 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:39.010 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:39.062 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:39.061+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:39.099 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 105.843697799, 2021-01-15T19:10:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 105.843532686, 2021-01-15T19:10:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 99.836126003999993, 2021-01-15T19:10:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.753389038999998, 2021-01-15T19:10:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.753239425000004, 2021-01-15T19:10:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.732941742999998, 2021-01-15T19:10:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:39.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:39.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.729885092000004, 2021-01-15T19:10:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.726871829999993, 2021-01-15T19:10:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.724124219000004, 2021-01-15T19:10:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.723690396999999, 2021-01-15T19:10:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.723566238000004, 2021-01-15T19:10:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 86.618889702999994, 2021-01-15T19:10:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 85.708950466999994, 2021-01-15T19:10:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:39.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:39.318 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.318 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.319 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:39.319 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:39.320 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.866229566000001, 2021-01-15T19:10:39.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:39.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.320 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.321 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:39.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.321 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:39.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:39.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:39.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:39.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:39.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:39.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:39.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:39.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:39.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:39.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:39.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:39.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:39.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:39.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:39.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:39.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:39.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:39.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:39.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:39.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:39.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.333 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.334 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:39.334 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:39.334 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.891650102, 2021-01-15T19:10:39.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:39.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.335 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:39.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.336 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:39.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:39.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:39.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:39.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:39.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:39.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:39.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:39.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:39.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:39.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:39.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:39.344 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:39.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:39.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:39.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:39.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:39.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:39.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:39.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:39.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:39.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:39.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:39.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:39.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:39.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:39.350 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.350 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.351 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:39.351 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:39.352 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 28.891518440999999, 2021-01-15T19:10:39.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:39.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.353 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:39.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.354 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:39.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:39.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:39.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:39.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:39.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:39.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:39.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:39.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:39.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:39.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:39.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:39.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:39.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:39.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:39.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:39.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:39.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:39.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:39.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:39.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:39.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:39.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:39.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:39.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:39.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:39.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:39.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:39.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:39.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:39.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:39.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:39.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.369 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.370 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1672 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~6144000,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e167)", 2021-01-15T19:10:39.370 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.513472+0000", 2021-01-15T19:10:39.370 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.583487447, 2021-01-15T19:10:39.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023736568999999999, 2021-01-15T19:10:39.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.372 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1672 2021-01-15T19:10:39.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.372 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513472+0000", 2021-01-15T19:10:39.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513472+0000", 2021-01-15T19:10:39.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6097000000000002e-05 2021-01-15T19:10:39.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513528+0000", 2021-01-15T19:10:39.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.661e-06 2021-01-15T19:10:39.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513536+0000", 2021-01-15T19:10:39.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2021-01-15T19:10:39.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513550+0000", 2021-01-15T19:10:39.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1689999999999997e-06 2021-01-15T19:10:39.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513556+0000", 2021-01-15T19:10:39.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014363049999999999 2021-01-15T19:10:39.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.514992+0000", 2021-01-15T19:10:39.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1141000000000001e-05 2021-01-15T19:10:39.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.515073+0000", 2021-01-15T19:10:39.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022135564 2021-01-15T19:10:39.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.537209+0000", 2021-01-15T19:10:39.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023736568999999999 2021-01-15T19:10:39.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.382 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.382 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1673 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~6144000,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e167)", 2021-01-15T19:10:39.383 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.513609+0000", 2021-01-15T19:10:39.383 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.5833509110000001, 2021-01-15T19:10:39.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028079653999999999, 2021-01-15T19:10:39.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.384 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1673 2021-01-15T19:10:39.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.385 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.385 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513609+0000", 2021-01-15T19:10:39.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513609+0000", 2021-01-15T19:10:39.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6870999999999998e-05 2021-01-15T19:10:39.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.387 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513666+0000", 2021-01-15T19:10:39.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5220000000000004e-06 2021-01-15T19:10:39.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513672+0000", 2021-01-15T19:10:39.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3616e-05 2021-01-15T19:10:39.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513686+0000", 2021-01-15T19:10:39.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0019999999999997e-06 2021-01-15T19:10:39.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513691+0000", 2021-01-15T19:10:39.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001583112 2021-01-15T19:10:39.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.515274+0000", 2021-01-15T19:10:39.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121321 2021-01-15T19:10:39.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.515395+0000", 2021-01-15T19:10:39.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026293210000000001 2021-01-15T19:10:39.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.541688+0000", 2021-01-15T19:10:39.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028079653999999999 2021-01-15T19:10:39.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.394 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.395 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1674 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~6144000,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e167)", 2021-01-15T19:10:39.395 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.513732+0000", 2021-01-15T19:10:39.396 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.5832279979999999, 2021-01-15T19:10:39.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032831451999999997, 2021-01-15T19:10:39.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:39.396 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1674 2021-01-15T19:10:39.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.397 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513732+0000", 2021-01-15T19:10:39.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513732+0000", 2021-01-15T19:10:39.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2164999999999999e-05 2021-01-15T19:10:39.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513784+0000", 2021-01-15T19:10:39.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-06 2021-01-15T19:10:39.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513790+0000", 2021-01-15T19:10:39.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1747e-05 2021-01-15T19:10:39.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513802+0000", 2021-01-15T19:10:39.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5190000000000003e-06 2021-01-15T19:10:39.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513806+0000", 2021-01-15T19:10:39.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050619259999999996 2021-01-15T19:10:39.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.518868+0000", 2021-01-15T19:10:39.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114505 2021-01-15T19:10:39.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.518983+0000", 2021-01-15T19:10:39.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027580470999999999 2021-01-15T19:10:39.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.546563+0000", 2021-01-15T19:10:39.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032831451999999997 2021-01-15T19:10:39.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.407 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.407 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.408 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1676 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~458752] snapc 86=[86,84,83,82,7d,7b,7a,72,70,6f,6b,67,64] ondisk+write+known_if_redirected e167)", 2021-01-15T19:10:39.408 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.527635+0000", 2021-01-15T19:10:39.408 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 3.569324849, 2021-01-15T19:10:39.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026863965, 2021-01-15T19:10:39.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.409 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1676 2021-01-15T19:10:39.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.410 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527635+0000", 2021-01-15T19:10:39.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527635+0000", 2021-01-15T19:10:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026864400000000001 2021-01-15T19:10:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527903+0000", 2021-01-15T19:10:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0713000000000001e-05 2021-01-15T19:10:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.414 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527914+0000", 2021-01-15T19:10:39.414 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.504e-05 2021-01-15T19:10:39.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.414 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527929+0000", 2021-01-15T19:10:39.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5430000000000001e-06 2021-01-15T19:10:39.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527935+0000", 2021-01-15T19:10:39.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018745034000000001 2021-01-15T19:10:39.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.546680+0000", 2021-01-15T19:10:39.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3111999999999999e-05 2021-01-15T19:10:39.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.546773+0000", 2021-01-15T19:10:39.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060443699999999999 2021-01-15T19:10:39.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.547377+0000", 2021-01-15T19:10:39.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006880387 2021-01-15T19:10:39.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.554258+0000", 2021-01-15T19:10:39.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145993 2021-01-15T19:10:39.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.554404+0000", 2021-01-15T19:10:39.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5062000000000006e-05 2021-01-15T19:10:39.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.554499+0000", 2021-01-15T19:10:39.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026863965 2021-01-15T19:10:39.422 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.422 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.423 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.423 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:39.424 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:10:39.428 INFO:teuthology.orchestra.run.smithi001.stderr:Nothing to do for non-filestore backend 2021-01-15T19:10:39.429 INFO:tasks.rados.rados.0.smithi001.stdout:703: done (7 left) 2021-01-15T19:10:39.429 INFO:tasks.rados.rados.0.smithi001.stdout:705: done (6 left) 2021-01-15T19:10:39.429 INFO:tasks.rados.rados.0.smithi001.stdout:706: done (5 left) 2021-01-15T19:10:39.429 INFO:tasks.rados.rados.0.smithi001.stdout:707: done (4 left) 2021-01-15T19:10:39.429 INFO:tasks.rados.rados.0.smithi001.stdout:708: done (3 left) 2021-01-15T19:10:39.430 INFO:tasks.rados.rados.0.smithi001.stdout:709: done (2 left) 2021-01-15T19:10:39.430 INFO:tasks.rados.rados.0.smithi001.stdout:710: done (1 left) 2021-01-15T19:10:39.430 INFO:tasks.rados.rados.0.smithi001.stdout:711: done (0 left) 2021-01-15T19:10:39.430 INFO:tasks.rados.rados.0.smithi001.stdout:712: read oid 4 snap -1 2021-01-15T19:10:39.430 INFO:tasks.rados.rados.0.smithi001.stdout:712: expect (ObjNum 241 snap 75 seq_num 241) 2021-01-15T19:10:39.431 INFO:tasks.rados.rados.0.smithi001.stdout:713: delete oid 33 current snap is 78 2021-01-15T19:10:39.431 INFO:tasks.rados.rados.0.smithi001.stdout:712: expect (ObjNum 241 snap 75 seq_num 241) 2021-01-15T19:10:39.431 INFO:tasks.rados.rados.0.smithi001.stdout:712: done (1 left) 2021-01-15T19:10:39.431 INFO:tasks.rados.rados.0.smithi001.stdout:713: done (0 left) 2021-01-15T19:10:39.431 INFO:tasks.rados.rados.0.smithi001.stdout:714: rmattr oid 39 current snap is 78 2021-01-15T19:10:39.432 INFO:tasks.rados.rados.0.smithi001.stdout:714: done (0 left) 2021-01-15T19:10:39.432 INFO:tasks.rados.rados.0.smithi001.stdout:715: delete oid 42 current snap is 78 2021-01-15T19:10:39.432 INFO:tasks.rados.rados.0.smithi001.stdout:715: done (0 left) 2021-01-15T19:10:39.432 INFO:tasks.rados.rados.0.smithi001.stdout:716: copy_from oid 31 from oid 25 current snap is 78 2021-01-15T19:10:39.432 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 49 current snap is 78 2021-01-15T19:10:39.433 INFO:tasks.rados.rados.0.smithi001.stdout:717: seq_num 249 ranges {4456448=573440,5029888=794624,5824512=409600} 2021-01-15T19:10:39.433 INFO:tasks.rados.rados.0.smithi001.stdout:717: writing smithi00140323-49 from 4456448 to 5029888 tid 1 2021-01-15T19:10:39.433 INFO:tasks.rados.rados.0.smithi001.stdout:717: writing smithi00140323-49 from 5029888 to 5824512 tid 2 2021-01-15T19:10:39.433 INFO:tasks.rados.rados.0.smithi001.stdout:717: writing smithi00140323-49 from 5824512 to 6234112 tid 3 2021-01-15T19:10:39.433 INFO:tasks.rados.rados.0.smithi001.stdout:718: rollback oid 34 current snap is 78 2021-01-15T19:10:39.434 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 34 to 75 2021-01-15T19:10:39.434 INFO:tasks.rados.rados.0.smithi001.stdout:716: finishing copy_from to smithi00140323-31 2021-01-15T19:10:39.434 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 277 (ObjNum 214 snap 68 seq_num 214) dirty exists 2021-01-15T19:10:39.434 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 35 current snap is 78 2021-01-15T19:10:39.434 INFO:tasks.rados.rados.0.smithi001.stdout:716: finishing copy_from racing read to smithi00140323-31 2021-01-15T19:10:39.435 INFO:tasks.rados.rados.0.smithi001.stdout:719: seq_num 250 ranges {0=589824,589824=679936,1269760=409600,1679360=114688} 2021-01-15T19:10:39.435 INFO:tasks.rados.rados.0.smithi001.stdout:719: writing smithi00140323-35 from 0 to 589824 tid 1 2021-01-15T19:10:39.435 INFO:tasks.rados.rados.0.smithi001.stdout:719: writing smithi00140323-35 from 589824 to 1269760 tid 2 2021-01-15T19:10:39.435 INFO:tasks.rados.rados.0.smithi001.stdout:719: writing smithi00140323-35 from 1269760 to 1679360 tid 3 2021-01-15T19:10:39.435 INFO:tasks.rados.rados.0.smithi001.stdout:719: writing smithi00140323-35 from 1679360 to 1794048 tid 4 2021-01-15T19:10:39.436 INFO:tasks.rados.rados.0.smithi001.stdout:716: done (3 left) 2021-01-15T19:10:39.436 INFO:tasks.rados.rados.0.smithi001.stdout:720: read oid 48 snap 74 2021-01-15T19:10:39.436 INFO:tasks.rados.rados.0.smithi001.stdout:720: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:10:39.436 INFO:tasks.rados.rados.0.smithi001.stdout:721: delete oid 28 current snap is 78 2021-01-15T19:10:39.436 INFO:tasks.rados.rados.0.smithi001.stdout:718: finishing rollback tid 1 to smithi00140323-34 2021-01-15T19:10:39.437 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 34 v 261 (ObjNum 178955104 snap 22026 seq_num 179163104) dirty dne 2021-01-15T19:10:39.437 INFO:tasks.rados.rados.0.smithi001.stdout:717: finishing write tid 1 to smithi00140323-49 2021-01-15T19:10:39.437 INFO:tasks.rados.rados.0.smithi001.stdout:717: finishing write tid 2 to smithi00140323-49 2021-01-15T19:10:39.437 INFO:tasks.rados.rados.0.smithi001.stdout:717: finishing write tid 3 to smithi00140323-49 2021-01-15T19:10:39.437 INFO:tasks.rados.rados.0.smithi001.stdout:717: finishing write tid 4 to smithi00140323-49 2021-01-15T19:10:39.438 INFO:tasks.rados.rados.0.smithi001.stdout:719: finishing write tid 1 to smithi00140323-35 2021-01-15T19:10:39.438 INFO:tasks.rados.rados.0.smithi001.stdout:717: finishing write tid 5 to smithi00140323-49 2021-01-15T19:10:39.438 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 49 v 169 (ObjNum 249 snap 78 seq_num 249) dirty exists 2021-01-15T19:10:39.438 INFO:tasks.rados.rados.0.smithi001.stdout:717: left oid 49 (ObjNum 249 snap 78 seq_num 249) 2021-01-15T19:10:39.438 INFO:tasks.rados.rados.0.smithi001.stdout:719: finishing write tid 2 to smithi00140323-35 2021-01-15T19:10:39.439 INFO:tasks.rados.rados.0.smithi001.stdout:719: finishing write tid 3 to smithi00140323-35 2021-01-15T19:10:39.439 INFO:tasks.rados.rados.0.smithi001.stdout:719: finishing write tid 4 to smithi00140323-35 2021-01-15T19:10:39.439 INFO:tasks.rados.rados.0.smithi001.stdout:717: done (4 left) 2021-01-15T19:10:39.439 INFO:tasks.rados.rados.0.smithi001.stdout:718: done (3 left) 2021-01-15T19:10:39.439 INFO:tasks.rados.rados.0.smithi001.stdout:721: done (2 left) 2021-01-15T19:10:39.440 INFO:tasks.rados.rados.0.smithi001.stdout:722: read oid 30 snap 76 2021-01-15T19:10:39.440 INFO:tasks.rados.rados.0.smithi001.stdout:722: expect deleted 2021-01-15T19:10:39.440 INFO:tasks.rados.rados.0.smithi001.stdout:723: delete oid 26 current snap is 78 2021-01-15T19:10:39.440 INFO:tasks.rados.rados.0.smithi001.stdout:719: finishing write tid 5 to smithi00140323-35 2021-01-15T19:10:39.440 INFO:tasks.rados.rados.0.smithi001.stdout:722: done (3 left) 2021-01-15T19:10:39.441 INFO:tasks.rados.rados.0.smithi001.stdout:723: done (2 left) 2021-01-15T19:10:39.441 INFO:tasks.rados.rados.0.smithi001.stdout:724: snap_create 2021-01-15T19:10:39.441 INFO:tasks.rados.rados.0.smithi001.stdout:719: finishing write tid 6 to smithi00140323-35 2021-01-15T19:10:39.441 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 197 (ObjNum 250 snap 78 seq_num 250) dirty exists 2021-01-15T19:10:39.441 INFO:tasks.rados.rados.0.smithi001.stdout:719: left oid 35 (ObjNum 250 snap 78 seq_num 250) 2021-01-15T19:10:39.442 INFO:tasks.rados.rados.0.smithi001.stdout:720: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:10:39.442 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:39.423+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:39.443 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:39.526 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:39.525+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:39.599 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:39.600 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:39.600 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:39.600 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:39.609 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:39.655 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:39.655+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:39.727 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:39.726+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:39.743 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:39.743 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:39.744 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:39.744 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:39.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:39.828 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:39.827+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:39.887 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:39.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.44999726, 2021-01-15T19:10:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:10:39.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:10:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:10:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:10:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:10:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:10:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:10:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:10:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:10:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:10:39.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:10:39.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:10:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:10:39.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:10:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:10:39.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:10:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:10:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:10:39.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:10:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:10:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:10:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:10:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:10:39.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:10:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:10:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:10:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:10:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:10:39.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:10:39.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:10:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.44985796, 2021-01-15T19:10:39.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:10:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:10:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.916 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:10:39.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:10:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:10:39.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:10:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:10:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:10:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:10:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:10:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:10:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:10:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:10:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:10:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:10:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:10:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:10:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:10:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:10:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:10:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:10:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:10:39.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:10:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:10:39.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:10:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:10:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:10:39.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:10:39.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:10:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:10:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:39.940 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.44973685399999, 2021-01-15T19:10:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:10:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.941 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:10:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:39.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:10:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:10:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:10:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:10:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:10:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:10:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:10:39.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:10:39.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:10:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:10:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:10:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:10:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:10:39.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:10:39.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:10:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:39.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:10:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:10:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:10:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:10:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:10:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:10:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:10:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:10:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:10:39.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:10:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:10:39.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:10:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:10:39.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:10:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 115.67945598, 2021-01-15T19:10:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:10:39.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:10:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:10:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:10:39.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:10:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:10:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:10:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:10:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:10:39.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:39.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:10:39.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:10:39.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:10:39.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:10:39.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:39.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:10:39.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:10:39.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:39.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:10:39.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:10:39.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:10:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:10:39.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:39.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:10:39.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:10:39.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:39.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:10:39.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:10:39.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:39.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:10:39.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:10:39.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:39.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:10:39.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:10:39.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.988 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:39.989 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:39.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.990 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:39.990 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:39.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 115.67047780199999, 2021-01-15T19:10:39.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:10:39.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:39.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:39.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:39.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:39.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:39.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:10:39.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.994 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:39.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:39.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:39.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:39.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:39.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:39.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:10:39.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:39.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:39.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:10:40.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:10:40.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:10:40.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:10:40.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:10:40.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:40.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:10:40.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:10:40.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:40.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:10:40.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:40.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:10:40.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:10:40.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:10:40.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:10:40.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:10:40.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:10:40.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:10:40.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:10:40.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:10:40.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:10:40.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:10:40.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:10:40.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:10:40.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:10:40.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.016 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.016 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.017 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:40.017 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 115.669958102, 2021-01-15T19:10:40.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:10:40.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.018 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:10:40.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.019 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:40.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:40.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:10:40.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:10:40.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:10:40.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:10:40.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:10:40.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:10:40.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:10:40.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:10:40.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:10:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:10:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:10:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:10:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:10:40.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:10:40.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:10:40.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:10:40.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:10:40.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:10:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:10:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:10:40.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:10:40.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:10:40.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:10:40.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:10:40.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:10:40.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.033 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.034 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.034 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:40.034 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 115.668277148, 2021-01-15T19:10:40.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:10:40.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.035 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.036 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:10:40.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.036 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:40.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:40.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:10:40.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:10:40.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:10:40.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:10:40.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:10:40.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:10:40.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:10:40.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:10:40.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:10:40.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:40.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:10:40.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:40.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:10:40.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:10:40.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:10:40.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:10:40.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:10:40.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:10:40.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:10:40.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:10:40.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:10:40.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:10:40.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:10:40.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:10:40.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:10:40.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:10:40.050 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.050 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.051 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:40.051 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:40.052 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 115.66814419400001, 2021-01-15T19:10:40.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:10:40.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.052 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.053 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:10:40.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.053 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:40.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:40.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:10:40.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:10:40.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:10:40.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:10:40.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:40.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:10:40.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:10:40.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:10:40.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:10:40.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:10:40.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:10:40.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:40.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:10:40.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:10:40.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:10:40.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:10:40.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:10:40.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:10:40.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:10:40.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:10:40.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:10:40.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:10:40.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:10:40.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:10:40.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.068 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:40.068 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:40.068 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.70561292399999, 2021-01-15T19:10:40.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:10:40.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.069 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:10:40.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.070 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:40.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:10:40.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:10:40.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:10:40.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:10:40.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:10:40.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:10:40.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:10:40.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:10:40.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:10:40.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:10:40.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:10:40.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:40.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:10:40.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:10:40.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:10:40.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:10:40.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:10:40.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:10:40.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:10:40.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:10:40.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:10:40.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:10:40.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:10:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:10:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:10:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:10:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.705446331, 2021-01-15T19:10:40.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:10:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:10:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:10:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:10:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:10:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:10:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:10:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:10:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:10:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:10:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:10:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:10:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:10:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:10:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:10:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:10:40.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:10:40.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:40.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:10:40.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:10:40.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:10:40.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:10:40.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:10:40.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:10:40.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:10:40.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:10:40.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:10:40.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:10:40.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:10:40.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:10:40.102 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.103 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.103 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.103 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:40.104 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:40.104 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.506907537000004, 2021-01-15T19:10:40.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:10:40.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.105 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:10:40.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.106 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:40.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:40.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:10:40.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:10:40.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:10:40.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:10:40.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:10:40.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:10:40.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:10:40.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:10:40.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:10:40.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:10:40.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:10:40.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:10:40.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:10:40.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:10:40.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:10:40.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:10:40.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:10:40.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:10:40.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:10:40.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:10:40.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:10:40.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:10:40.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:10:40.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:10:40.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:10:40.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.121 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:40.121 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:40.121 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.504637492000001, 2021-01-15T19:10:40.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:10:40.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:10:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:40.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:40.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:10:40.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:10:40.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:10:40.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:10:40.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:10:40.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:10:40.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:10:40.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:10:40.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:10:40.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:10:40.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:10:40.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:10:40.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:10:40.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:10:40.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:10:40.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:10:40.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:10:40.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:10:40.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:10:40.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:10:40.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:10:40.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:10:40.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:10:40.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:10:40.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:10:40.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:40.139 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:40.139 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.500099301000006, 2021-01-15T19:10:40.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:10:40.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:10:40.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.141 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:40.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:40.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:10:40.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:10:40.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:10:40.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:10:40.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:10:40.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:10:40.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:40.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:10:40.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:10:40.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:10:40.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:10:40.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:10:40.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:10:40.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:10:40.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:10:40.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:10:40.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:10:40.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:10:40.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:10:40.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:10:40.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:10:40.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:10:40.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:10:40.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:10:40.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:10:40.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.155 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.156 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:40.156 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:40.156 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.497513428999994, 2021-01-15T19:10:40.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:10:40.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:10:40.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.158 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:40.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:40.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:10:40.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:10:40.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:10:40.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:10:40.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:10:40.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:10:40.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:10:40.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:10:40.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:10:40.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:10:40.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:10:40.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:10:40.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:10:40.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:10:40.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:10:40.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:10:40.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:10:40.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:10:40.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:10:40.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:10:40.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:10:40.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:10:40.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:10:40.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:10:40.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:10:40.172 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.172 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.172 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.173 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:40.173 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:40.173 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.497139094000005, 2021-01-15T19:10:40.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:10:40.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.174 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.175 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:10:40.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.175 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:40.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:40.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:10:40.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:10:40.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:10:40.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:10:40.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:10:40.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:10:40.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:10:40.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:10:40.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:10:40.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:10:40.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:10:40.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:10:40.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:10:40.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:10:40.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:10:40.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:10:40.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:10:40.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:10:40.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:10:40.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:10:40.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:10:40.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:10:40.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:10:40.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:10:40.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:10:40.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.190 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.191 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:40.191 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:40.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 87.497002598999998, 2021-01-15T19:10:40.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:10:40.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.192 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:10:40.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.193 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:40.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:40.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:10:40.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:10:40.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:10:40.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:10:40.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:10:40.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:10:40.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:10:40.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:10:40.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:10:40.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:10:40.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:10:40.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:10:40.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:10:40.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:10:40.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:10:40.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:10:40.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:10:40.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:10:40.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:10:40.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.206 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:10:40.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:10:40.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.207 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:10:40.208 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:10:40.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.208 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.209 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:10:40.209 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:40.209 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 85.514604879000004, 2021-01-15T19:10:40.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:10:40.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.211 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:10:40.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.211 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:40.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:40.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:10:40.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:10:40.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:10:40.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:10:40.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:10:40.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:10:40.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:10:40.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:10:40.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:10:40.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:10:40.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:10:40.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:40.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:10:40.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:10:40.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:10:40.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:10:40.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:10:40.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:10:40.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:10:40.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:10:40.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.224 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.225 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:10:40.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:10:40.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.225 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:10:40.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:10:40.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.227 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:10:40.227 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:40.228 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.367851820999999, 2021-01-15T19:10:40.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:10:40.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.228 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:10:40.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.230 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:40.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:40.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:10:40.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:10:40.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:10:40.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:10:40.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:10:40.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:10:40.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:10:40.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:10:40.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:10:40.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:10:40.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:10:40.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:10:40.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:10:40.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:10:40.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:10:40.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:10:40.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:10:40.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.241 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:10:40.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:10:40.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:10:40.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:10:40.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:10:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 18.367688889, 2021-01-15T19:10:40.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:10:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.245 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:10:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:10:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:10:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:10:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:10:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:10:40.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:10:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:10:40.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:10:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:10:40.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:10:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:10:40.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:10:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:10:40.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:10:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:10:40.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:10:40.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:10:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:10:40.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:10:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:10:40.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:10:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:10:40.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:10:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:10:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 12.594016251999999, 2021-01-15T19:10:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:10:40.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:10:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:10:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:10:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:10:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:10:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:10:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:10:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:10:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:10:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:10:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:10:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:10:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:10:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:10:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:10:40.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:10:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:10:40.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:10:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:10:40.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.274 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.275 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:40.277 INFO:tasks.rados.rados.0.smithi001.stdout:719: done (2 left) 2021-01-15T19:10:40.278 INFO:tasks.rados.rados.0.smithi001.stdout:720: done (1 left) 2021-01-15T19:10:40.278 INFO:tasks.rados.rados.0.smithi001.stdout:724: done (0 left) 2021-01-15T19:10:40.278 INFO:tasks.rados.rados.0.smithi001.stdout:725: read oid 22 snap 55 2021-01-15T19:10:40.278 INFO:tasks.rados.rados.0.smithi001.stdout:725: expect (ObjNum 158 snap 41 seq_num 158) 2021-01-15T19:10:40.279 INFO:tasks.rados.rados.0.smithi001.stdout:726: rollback oid 28 current snap is 79 2021-01-15T19:10:40.279 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 28 to 62 2021-01-15T19:10:40.279 INFO:tasks.rados.rados.0.smithi001.stdout:727: read oid 34 snap -1 2021-01-15T19:10:40.279 INFO:tasks.rados.rados.0.smithi001.stdout:727: expect deleted 2021-01-15T19:10:40.280 INFO:tasks.rados.rados.0.smithi001.stdout:728: delete oid 44 current snap is 79 2021-01-15T19:10:40.280 INFO:tasks.rados.rados.0.smithi001.stdout:727: done (3 left) 2021-01-15T19:10:40.280 INFO:tasks.rados.rados.0.smithi001.stdout:728: done (2 left) 2021-01-15T19:10:40.281 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 44 current snap is 79 2021-01-15T19:10:40.281 INFO:tasks.rados.rados.0.smithi001.stdout:729: seq_num 254 ranges {0=663552,663552=663552,1327104=540672} 2021-01-15T19:10:40.281 INFO:tasks.rados.rados.0.smithi001.stdout:729: writing smithi00140323-44 from 0 to 663552 tid 1 2021-01-15T19:10:40.281 INFO:tasks.rados.rados.0.smithi001.stdout:729: writing smithi00140323-44 from 663552 to 1327104 tid 2 2021-01-15T19:10:40.281 INFO:tasks.rados.rados.0.smithi001.stdout:729: writing smithi00140323-44 from 1327104 to 1867776 tid 3 2021-01-15T19:10:40.282 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 45 current snap is 79 2021-01-15T19:10:40.282 INFO:tasks.rados.rados.0.smithi001.stdout:725: expect (ObjNum 158 snap 41 seq_num 158) 2021-01-15T19:10:40.282 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:10:40.283 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:40.275+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:40.283 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:40.295 INFO:tasks.rados.rados.0.smithi001.stdout:726: finishing rollback tid 1 to smithi00140323-28 2021-01-15T19:10:40.296 INFO:tasks.rados.rados.0.smithi001.stdout:726: finishing rollback tid 2 to smithi00140323-28 2021-01-15T19:10:40.296 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 28 v 348 (ObjNum 167 snap 47 seq_num 167) dirty exists 2021-01-15T19:10:40.296 INFO:tasks.rados.rados.0.smithi001.stdout:729: finishing write tid 1 to smithi00140323-44 2021-01-15T19:10:40.296 INFO:tasks.rados.rados.0.smithi001.stdout:729: finishing write tid 2 to smithi00140323-44 2021-01-15T19:10:40.296 INFO:tasks.rados.rados.0.smithi001.stdout:730: seq_num 255 ranges {2244608=679936,2924544=548864,3473408=598016,4071424=57344} 2021-01-15T19:10:40.311 INFO:tasks.rados.rados.0.smithi001.stdout:730: writing smithi00140323-45 from 2244608 to 2924544 tid 1 2021-01-15T19:10:40.314 INFO:tasks.rados.rados.0.smithi001.stdout:730: writing smithi00140323-45 from 2924544 to 3473408 tid 2 2021-01-15T19:10:40.318 INFO:tasks.rados.rados.0.smithi001.stdout:730: writing smithi00140323-45 from 3473408 to 4071424 tid 3 2021-01-15T19:10:40.319 INFO:tasks.rados.rados.0.smithi001.stdout:730: writing smithi00140323-45 from 4071424 to 4128768 tid 4 2021-01-15T19:10:40.319 INFO:tasks.rados.rados.0.smithi001.stdout:725: done (3 left) 2021-01-15T19:10:40.319 INFO:tasks.rados.rados.0.smithi001.stdout:726: done (2 left) 2021-01-15T19:10:40.319 INFO:tasks.rados.rados.0.smithi001.stdout:731: copy_from oid 18 from oid 34 current snap is 79 2021-01-15T19:10:40.320 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 30 current snap is 79 2021-01-15T19:10:40.320 INFO:tasks.rados.rados.0.smithi001.stdout:732: seq_num 256 ranges {0=696320,696320=434176,1130496=770048} 2021-01-15T19:10:40.323 INFO:tasks.rados.rados.0.smithi001.stdout:732: writing smithi00140323-30 from 0 to 696320 tid 1 2021-01-15T19:10:40.326 INFO:tasks.rados.rados.0.smithi001.stdout:732: writing smithi00140323-30 from 696320 to 1130496 tid 2 2021-01-15T19:10:40.331 INFO:tasks.rados.rados.0.smithi001.stdout:732: writing smithi00140323-30 from 1130496 to 1900544 tid 3 2021-01-15T19:10:40.331 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 19 current snap is 79 2021-01-15T19:10:40.331 INFO:tasks.rados.rados.0.smithi001.stdout:733: seq_num 257 ranges {3702784=712704,4415488=720896,5136384=483328} 2021-01-15T19:10:40.355 INFO:tasks.rados.rados.0.smithi001.stdout:733: writing smithi00140323-19 from 3702784 to 4415488 tid 1 2021-01-15T19:10:40.360 INFO:tasks.rados.rados.0.smithi001.stdout:733: writing smithi00140323-19 from 4415488 to 5136384 tid 2 2021-01-15T19:10:40.364 INFO:tasks.rados.rados.0.smithi001.stdout:733: writing smithi00140323-19 from 5136384 to 5619712 tid 3 2021-01-15T19:10:40.365 INFO:tasks.rados.rados.0.smithi001.stdout:734: read oid 3 snap 64 2021-01-15T19:10:40.365 INFO:tasks.rados.rados.0.smithi001.stdout:734: expect (ObjNum 170 snap 49 seq_num 170) 2021-01-15T19:10:40.365 INFO:tasks.rados.rados.0.smithi001.stdout:735: snap_remove snap 76 2021-01-15T19:10:40.376 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:40.376+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:40.426 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:40.427 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:40.427 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:40.427 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:40.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:40.477 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:40.476+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:40.579 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:40.578+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:40.608 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:40.608 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:40.609 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:40.609 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:40.609 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:40.618 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:40.680 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:40.679+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:40.752 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:40.752 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.500881208, 2021-01-15T19:10:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:10:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:10:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:10:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:10:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:10:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:10:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:10:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:10:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:10:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:10:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:10:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:10:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:10:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:10:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:10:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:10:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:10:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:10:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:10:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:10:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:10:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:10:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:10:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.500690264, 2021-01-15T19:10:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:10:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:10:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:10:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:10:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:10:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:10:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:10:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:10:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:10:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:10:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:10:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:10:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:10:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:10:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:10:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:10:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:10:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:10:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:10:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:10:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:10:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:10:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:10:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:10:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:10:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.561738167, 2021-01-15T19:10:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:10:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:10:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:10:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:10:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:10:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:10:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:10:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:10:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:10:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:10:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:10:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:10:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:10:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:10:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:10:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:10:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:10:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:10:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:10:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:10:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:10:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:10:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:10:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.55901770600001, 2021-01-15T19:10:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:10:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:10:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:10:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:10:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:10:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:10:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:10:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:10:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:10:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:10:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:10:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:10:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:10:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:10:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:10:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:10:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:10:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:10:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:10:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:10:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:10:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:10:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:10:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:10:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:10:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.558905127, 2021-01-15T19:10:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:10:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:10:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:10:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:10:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:10:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:10:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:10:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:10:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:10:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:10:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:10:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:10:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:10:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:10:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:10:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:10:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:10:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:10:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:10:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:10:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:10:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:10:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:10:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.558775223, 2021-01-15T19:10:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:10:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:10:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:10:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:10:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:10:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:10:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:10:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:10:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:10:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:10:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:10:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:10:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:10:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:10:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:10:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:10:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:10:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:10:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:10:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:10:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:10:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:10:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:10:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:10:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:10:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.535569954, 2021-01-15T19:10:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:10:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:10:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:10:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:10:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:10:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:10:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:10:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:10:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:10:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:10:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:10:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:10:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:10:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:10:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:10:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:10:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:10:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:10:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:10:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:10:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:10:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.535451756, 2021-01-15T19:10:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:10:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:10:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:10:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:10:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:10:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:10:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:10:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:10:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:10:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:10:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:10:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:10:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:10:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:10:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:10:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:10:40.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:10:40.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:10:40.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:10:40.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:10:40.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:10:40.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:10:40.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:10:40.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:40.886 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.53533906600001, 2021-01-15T19:10:40.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:10:40.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.888 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:10:40.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.888 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:40.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:40.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:10:40.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:10:40.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:10:40.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:10:40.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:10:40.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:10:40.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:10:40.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:10:40.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:10:40.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:10:40.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:10:40.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:10:40.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:10:40.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:10:40.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:10:40.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:10:40.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:10:40.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:10:40.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:10:40.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:10:40.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:10:40.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.902 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.903 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:10:40.903 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:40.903 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.52973731599999, 2021-01-15T19:10:40.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:10:40.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:40.904 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:40.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:40.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:10:40.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:10:40.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:10:40.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:10:40.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:10:40.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:10:40.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:10:40.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:10:40.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:10:40.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:10:40.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:10:40.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:10:40.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:10:40.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.913 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.914 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.914 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:40.915 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.521268592, 2021-01-15T19:10:40.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:10:40.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.915 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:10:40.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.916 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:40.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:40.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:10:40.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:10:40.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:10:40.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:10:40.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:10:40.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:10:40.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:10:40.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:10:40.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:10:40.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:10:40.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:10:40.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:10:40.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:10:40.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:10:40.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:10:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:10:40.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:10:40.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:10:40.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:10:40.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:10:40.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:10:40.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.930 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:40.931 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:40.932 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.51675000900001, 2021-01-15T19:10:40.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:10:40.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:10:40.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:40.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:40.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:10:40.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:10:40.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:10:40.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:10:40.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:10:40.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:10:40.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:10:40.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:10:40.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:10:40.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:10:40.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:10:40.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:10:40.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:10:40.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:10:40.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:10:40.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:10:40.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:10:40.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:10:40.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:10:40.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:10:40.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:10:40.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.948 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.949 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:40.949 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:40.949 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 116.51173224, 2021-01-15T19:10:40.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:10:40.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:40.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:10:40.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.951 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:10:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:10:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:10:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:10:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:10:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:10:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:10:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:10:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:10:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:10:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:10:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:10:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:10:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:10:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:10:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:10:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:10:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:10:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:10:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:10:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 102.544608162, 2021-01-15T19:10:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:10:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:10:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:40.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:40.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:10:40.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:10:40.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:10:40.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:10:40.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:10:40.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:10:40.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:10:40.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:10:40.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:10:40.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:10:40.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:10:40.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:40.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:10:40.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:10:40.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:10:40.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:10:40.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:10:40.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:10:40.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:10:40.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:10:40.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:40.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:10:40.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:10:40.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:40.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:10:40.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:10:40.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:40.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:10:40.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:10:40.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.983 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:40.984 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:40.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.984 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:10:40.985 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:40.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.567421514000003, 2021-01-15T19:10:40.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:10:40.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:40.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:40.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:40.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:40.986 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:40.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:10:40.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.987 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:40.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:40.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:40.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:40.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:40.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:40.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:10:40.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:40.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:10:40.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:10:40.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:40.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:10:40.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:10:40.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:40.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:10:40.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:10:40.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:40.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:10:40.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:10:40.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:10:40.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:40.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:40.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:10:40.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:10:40.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:40.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:10:40.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:10:40.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:40.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:10:40.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:10:40.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:40.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:40.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:40.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:10:40.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:10:40.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:41.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:10:41.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:10:41.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:41.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:10:41.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:10:41.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:41.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:10:41.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:10:41.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.004 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:10:41.004 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:41.004 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 42.536874492999999, 2021-01-15T19:10:41.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:10:41.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:41.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:41.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:41.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:41.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:41.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:10:41.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.007 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:41.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:41.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:41.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:41.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:41.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:41.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:10:41.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:41.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:10:41.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:10:41.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:41.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:10:41.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:10:41.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:41.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:10:41.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:10:41.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:41.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:10:41.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:10:41.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:10:41.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:10:41.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:41.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:10:41.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:10:41.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:10:41.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:10:41.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:41.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:10:41.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:10:41.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:10:41.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:10:41.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:41.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:10:41.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:10:41.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.022 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.022 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:41.023 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:10:41.023 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:10:41.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.023 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.023 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.024 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:41.024 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:41.025 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.546216518000001, 2021-01-15T19:10:41.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:10:41.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:41.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:41.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:41.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:41.026 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:41.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:10:41.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.027 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:41.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:41.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:41.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:41.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:41.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:41.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:10:41.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:41.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:41.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:10:41.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:41.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:10:41.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:10:41.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:41.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:10:41.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:10:41.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:41.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:10:41.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:10:41.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:10:41.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:10:41.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:41.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:10:41.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:10:41.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:10:41.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:10:41.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:41.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:10:41.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:10:41.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:41.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:10:41.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:10:41.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:41.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:10:41.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:10:41.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:41.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:10:41.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:10:41.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:41.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:10:41.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:10:41.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.043 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.043 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.044 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:41.044 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:41.044 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.546076984999999, 2021-01-15T19:10:41.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:10:41.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:41.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:41.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:41.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:41.046 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:41.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:10:41.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.046 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:41.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:41.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:41.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:41.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:41.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:41.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:10:41.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:41.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:10:41.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:10:41.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:41.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:10:41.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:10:41.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:41.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:10:41.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:10:41.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:41.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:10:41.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:10:41.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:10:41.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:10:41.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:41.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:10:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:10:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:10:41.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:10:41.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:41.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:10:41.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:10:41.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:41.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:10:41.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:10:41.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:41.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:10:41.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:10:41.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:41.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:10:41.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:10:41.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:41.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:10:41.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:10:41.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.064 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.065 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:41.065 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:41.065 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.545961671999997, 2021-01-15T19:10:41.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:10:41.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:41.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:41.066 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:41.066 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:41.066 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:41.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:10:41.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.067 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:41.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:41.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:41.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:41.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:41.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:41.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:10:41.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:41.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:10:41.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:10:41.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:41.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:10:41.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:10:41.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:41.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:10:41.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:10:41.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:41.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:41.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:10:41.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:10:41.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:10:41.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:41.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:10:41.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:10:41.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:10:41.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:10:41.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:41.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:10:41.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:10:41.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:41.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:10:41.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:10:41.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:41.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:10:41.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:10:41.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:41.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:10:41.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:10:41.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:41.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:10:41.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:10:41.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.083 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.083 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.084 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:10:41.084 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:41.085 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 1.4324903760000001, 2021-01-15T19:10:41.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:10:41.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:41.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:41.085 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:41.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:41.086 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:41.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:10:41.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.087 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:41.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:41.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:41.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:41.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:41.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:41.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:10:41.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:41.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:10:41.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:10:41.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:41.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:10:41.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:10:41.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:41.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:10:41.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:10:41.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:41.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:10:41.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:10:41.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:41.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:10:41.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:10:41.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:41.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:10:41.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:10:41.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:41.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:41.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:41.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:10:41.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:10:41.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:41.098 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:41.098 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:41.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:41.100 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:41.098+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:41.101 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:41.201 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:41.200+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:41.242 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:41.243 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:41.244 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:41.246 INFO:tasks.rados.rados.0.smithi001.stdout:735: done (6 left) 2021-01-15T19:10:41.246 INFO:tasks.rados.rados.0.smithi001.stdout:736: read oid 26 snap -1 2021-01-15T19:10:41.246 INFO:tasks.rados.rados.0.smithi001.stdout:736: expect deleted 2021-01-15T19:10:41.247 INFO:tasks.rados.rados.0.smithi001.stdout:737: delete oid 9 current snap is 79 2021-01-15T19:10:41.247 INFO:tasks.rados.rados.0.smithi001.stdout:729: finishing write tid 3 to smithi00140323-44 2021-01-15T19:10:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:729: finishing write tid 4 to smithi00140323-44 2021-01-15T19:10:41.248 INFO:tasks.rados.rados.0.smithi001.stdout:729: finishing write tid 5 to smithi00140323-44 2021-01-15T19:10:41.249 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 44 v 97 (ObjNum 254 snap 79 seq_num 254) dirty exists 2021-01-15T19:10:41.249 INFO:tasks.rados.rados.0.smithi001.stdout:729: left oid 44 (ObjNum 254 snap 79 seq_num 254) 2021-01-15T19:10:41.249 INFO:tasks.rados.rados.0.smithi001.stdout:730: finishing write tid 1 to smithi00140323-45 2021-01-15T19:10:41.250 INFO:tasks.rados.rados.0.smithi001.stdout:731: finishing copy_from to smithi00140323-18 2021-01-15T19:10:41.250 INFO:tasks.rados.rados.0.smithi001.stdout:731: got expected ENOENT (src dne) 2021-01-15T19:10:41.251 INFO:tasks.rados.rados.0.smithi001.stdout:731: finishing copy_from racing read to smithi00140323-18 2021-01-15T19:10:41.251 INFO:tasks.rados.rados.0.smithi001.stdout:730: finishing write tid 2 to smithi00140323-45 2021-01-15T19:10:41.252 INFO:tasks.rados.rados.0.smithi001.stdout:730: finishing write tid 3 to smithi00140323-45 2021-01-15T19:10:41.252 INFO:tasks.rados.rados.0.smithi001.stdout:730: finishing write tid 4 to smithi00140323-45 2021-01-15T19:10:41.253 INFO:tasks.rados.rados.0.smithi001.stdout:730: finishing write tid 5 to smithi00140323-45 2021-01-15T19:10:41.253 INFO:tasks.rados.rados.0.smithi001.stdout:732: finishing write tid 1 to smithi00140323-30 2021-01-15T19:10:41.253 INFO:tasks.rados.rados.0.smithi001.stdout:732: finishing write tid 2 to smithi00140323-30 2021-01-15T19:10:41.254 INFO:tasks.rados.rados.0.smithi001.stdout:730: finishing write tid 6 to smithi00140323-45 2021-01-15T19:10:41.254 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 361 (ObjNum 255 snap 79 seq_num 255) dirty exists 2021-01-15T19:10:41.255 INFO:tasks.rados.rados.0.smithi001.stdout:730: left oid 45 (ObjNum 255 snap 79 seq_num 255) 2021-01-15T19:10:41.255 INFO:tasks.rados.rados.0.smithi001.stdout:732: finishing write tid 3 to smithi00140323-30 2021-01-15T19:10:41.255 INFO:tasks.rados.rados.0.smithi001.stdout:732: finishing write tid 4 to smithi00140323-30 2021-01-15T19:10:41.256 INFO:tasks.rados.rados.0.smithi001.stdout:732: finishing write tid 5 to smithi00140323-30 2021-01-15T19:10:41.256 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 30 v 101 (ObjNum 256 snap 79 seq_num 256) dirty exists 2021-01-15T19:10:41.257 INFO:tasks.rados.rados.0.smithi001.stdout:732: left oid 30 (ObjNum 256 snap 79 seq_num 256) 2021-01-15T19:10:41.257 INFO:tasks.rados.rados.0.smithi001.stdout:733: finishing write tid 1 to smithi00140323-19 2021-01-15T19:10:41.257 INFO:tasks.rados.rados.0.smithi001.stdout:733: finishing write tid 2 to smithi00140323-19 2021-01-15T19:10:41.258 INFO:tasks.rados.rados.0.smithi001.stdout:733: finishing write tid 3 to smithi00140323-19 2021-01-15T19:10:41.258 INFO:tasks.rados.rados.0.smithi001.stdout:734: expect (ObjNum 170 snap 49 seq_num 170) 2021-01-15T19:10:41.260 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:41.275 INFO:tasks.rados.rados.0.smithi001.stdout:733: finishing write tid 4 to smithi00140323-19 2021-01-15T19:10:41.275 INFO:tasks.rados.rados.0.smithi001.stdout:733: finishing write tid 5 to smithi00140323-19 2021-01-15T19:10:41.275 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 19 v 193 (ObjNum 257 snap 79 seq_num 257) dirty exists 2021-01-15T19:10:41.276 INFO:tasks.rados.rados.0.smithi001.stdout:733: left oid 19 (ObjNum 257 snap 79 seq_num 257) 2021-01-15T19:10:41.276 INFO:tasks.rados.rados.0.smithi001.stdout:729: done (7 left) 2021-01-15T19:10:41.276 INFO:tasks.rados.rados.0.smithi001.stdout:730: done (6 left) 2021-01-15T19:10:41.276 INFO:tasks.rados.rados.0.smithi001.stdout:731: done (5 left) 2021-01-15T19:10:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:732: done (4 left) 2021-01-15T19:10:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:733: done (3 left) 2021-01-15T19:10:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:734: done (2 left) 2021-01-15T19:10:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:737: done (1 left) 2021-01-15T19:10:41.277 INFO:tasks.rados.rados.0.smithi001.stdout:738: read oid 22 snap -1 2021-01-15T19:10:41.278 INFO:tasks.rados.rados.0.smithi001.stdout:738: expect (ObjNum 240 snap 75 seq_num 240) 2021-01-15T19:10:41.278 INFO:tasks.rados.rados.0.smithi001.stdout:739: snap_remove snap 62 2021-01-15T19:10:41.302 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:41.300+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:41.302 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:41.322 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:41.342 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:41.363 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:41.384 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:41.402 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:41.401+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:41.405 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:41.425 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.428 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.428 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:41.429 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:41.429 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:41.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:41.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:41.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:41.504 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:41.502+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:41.604 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:41.603+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:41.605 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:41.605 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:41.615 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:41.706 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:41.704+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:41.791 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:41.835 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:41.834+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:41.907 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:41.906+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:41.916 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:41.917 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:41.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:42.008 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:42.006+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:42.092 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:42.092 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:42.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:42.146 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:42.145+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:42.209 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:42.208+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:42.234 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:42.234 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:42.244 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:42.252 INFO:tasks.rados.rados.0.smithi001.stdout:739: done (2 left) 2021-01-15T19:10:42.252 INFO:tasks.rados.rados.0.smithi001.stdout:740: rmattr oid 13 current snap is 79 2021-01-15T19:10:42.252 INFO:tasks.rados.rados.0.smithi001.stdout:740: done (2 left) 2021-01-15T19:10:42.252 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 50 current snap is 79 2021-01-15T19:10:42.253 INFO:tasks.rados.rados.0.smithi001.stdout:741: seq_num 259 ranges {4841472=745472,5586944=540672,6127616=581632,6709248=81920} 2021-01-15T19:10:42.283 INFO:tasks.rados.rados.0.smithi001.stdout:741: writing smithi00140323-50 from 4841472 to 5586944 tid 1 2021-01-15T19:10:42.285 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:10:42.287 INFO:tasks.rados.rados.0.smithi001.stdout:741: writing smithi00140323-50 from 5586944 to 6127616 tid 2 2021-01-15T19:10:42.289 INFO:tasks.rados.rados.0.smithi001.stdout:741: writing smithi00140323-50 from 6127616 to 6709248 tid 3 2021-01-15T19:10:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:741: writing smithi00140323-50 from 6709248 to 6791168 tid 4 2021-01-15T19:10:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:742: setattr oid 42 current snap is 79 2021-01-15T19:10:42.291 INFO:tasks.rados.rados.0.smithi001.stdout:738: expect (ObjNum 240 snap 75 seq_num 240) 2021-01-15T19:10:42.330 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:42.329+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:42.386 INFO:tasks.rados.rados.0.smithi001.stdout:741: finishing write tid 1 to smithi00140323-50 2021-01-15T19:10:42.386 INFO:tasks.rados.rados.0.smithi001.stdout:741: finishing write tid 2 to smithi00140323-50 2021-01-15T19:10:42.386 INFO:tasks.rados.rados.0.smithi001.stdout:741: finishing write tid 3 to smithi00140323-50 2021-01-15T19:10:42.387 INFO:tasks.rados.rados.0.smithi001.stdout:741: finishing write tid 4 to smithi00140323-50 2021-01-15T19:10:42.387 INFO:tasks.rados.rados.0.smithi001.stdout:741: finishing write tid 5 to smithi00140323-50 2021-01-15T19:10:42.387 INFO:tasks.rados.rados.0.smithi001.stdout:741: finishing write tid 6 to smithi00140323-50 2021-01-15T19:10:42.387 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 50 v 261 (ObjNum 259 snap 79 seq_num 259) dirty exists 2021-01-15T19:10:42.387 INFO:tasks.rados.rados.0.smithi001.stdout:741: left oid 50 (ObjNum 259 snap 79 seq_num 259) 2021-01-15T19:10:42.388 INFO:tasks.rados.rados.0.smithi001.stdout:736: done (3 left) 2021-01-15T19:10:42.388 INFO:tasks.rados.rados.0.smithi001.stdout:738: done (2 left) 2021-01-15T19:10:42.388 INFO:tasks.rados.rados.0.smithi001.stdout:741: done (1 left) 2021-01-15T19:10:42.388 INFO:tasks.rados.rados.0.smithi001.stdout:743: read oid 8 snap -1 2021-01-15T19:10:42.389 INFO:tasks.rados.rados.0.smithi001.stdout:743: expect (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:42.389 INFO:tasks.rados.rados.0.smithi001.stdout:744: copy_from oid 6 from oid 16 current snap is 79 2021-01-15T19:10:42.389 INFO:tasks.rados.rados.0.smithi001.stdout:745: snap_remove snap 78 2021-01-15T19:10:42.411 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:42.409+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:42.419 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:42.419 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:42.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:42.513 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:42.512+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:42.595 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:42.595 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:42.605 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:42.646 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:42.645+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:42.712 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:42.711+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:42.776 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:42.777 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:42.787 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:42.814 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:42.812+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:42.915 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:42.914+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:42.952 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:42.952 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:42.961 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:43.046 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:43.045+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:43.117 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:43.116+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:43.127 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:43.127 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:43.136 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2021-01-15T19:10:43.218 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:43.217+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:43.257 INFO:tasks.rados.rados.0.smithi001.stdout:745: done (3 left) 2021-01-15T19:10:43.257 INFO:tasks.rados.rados.0.smithi001.stdout:746: delete oid 25 current snap is 79 2021-01-15T19:10:43.257 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 42 v 154 (ObjNum 179006960 snap 22026 seq_num 179064112) dirty exists 2021-01-15T19:10:43.258 INFO:tasks.rados.rados.0.smithi001.stdout:744: finishing copy_from to smithi00140323-6 2021-01-15T19:10:43.258 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 6 v 166 (ObjNum 232 snap 71 seq_num 232) dirty exists 2021-01-15T19:10:43.258 INFO:tasks.rados.rados.0.smithi001.stdout:744: finishing copy_from racing read to smithi00140323-6 2021-01-15T19:10:43.258 INFO:tasks.rados.rados.0.smithi001.stdout:743: expect (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:10:43.289 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:43.289 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:43.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2021-01-15T19:10:43.340 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:43.339+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:43.420 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:43.419+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:43.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:43.473 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:43.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2021-01-15T19:10:43.498 INFO:tasks.rados.rados.0.smithi001.stdout:742: done (3 left) 2021-01-15T19:10:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:743: done (2 left) 2021-01-15T19:10:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:744: done (1 left) 2021-01-15T19:10:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:746: done (0 left) 2021-01-15T19:10:43.499 INFO:tasks.rados.rados.0.smithi001.stdout:747: read oid 40 snap 61 2021-01-15T19:10:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:747: expect deleted 2021-01-15T19:10:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 13 current snap is 79 2021-01-15T19:10:43.500 INFO:tasks.rados.rados.0.smithi001.stdout:748: seq_num 261 ranges {6324224=778240,7102464=704512,7806976=499712} 2021-01-15T19:10:43.520 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:43.519+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:43.537 INFO:tasks.rados.rados.0.smithi001.stdout:748: writing smithi00140323-13 from 6324224 to 7102464 tid 1 2021-01-15T19:10:43.541 INFO:tasks.rados.rados.0.smithi001.stdout:748: writing smithi00140323-13 from 7102464 to 7806976 tid 2 2021-01-15T19:10:43.544 INFO:tasks.rados.rados.0.smithi001.stdout:748: writing smithi00140323-13 from 7806976 to 8306688 tid 3 2021-01-15T19:10:43.544 INFO:tasks.rados.rados.0.smithi001.stdout:749: snap_create 2021-01-15T19:10:43.554 INFO:tasks.rados.rados.0.smithi001.stdout:748: finishing write tid 1 to smithi00140323-13 2021-01-15T19:10:43.555 INFO:tasks.rados.rados.0.smithi001.stdout:748: finishing write tid 2 to smithi00140323-13 2021-01-15T19:10:43.555 INFO:tasks.rados.rados.0.smithi001.stdout:748: finishing write tid 3 to smithi00140323-13 2021-01-15T19:10:43.556 INFO:tasks.rados.rados.0.smithi001.stdout:748: finishing write tid 4 to smithi00140323-13 2021-01-15T19:10:43.557 INFO:tasks.rados.rados.0.smithi001.stdout:748: finishing write tid 5 to smithi00140323-13 2021-01-15T19:10:43.557 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 13 v 363 (ObjNum 261 snap 79 seq_num 261) dirty exists 2021-01-15T19:10:43.557 INFO:tasks.rados.rados.0.smithi001.stdout:748: left oid 13 (ObjNum 261 snap 79 seq_num 261) 2021-01-15T19:10:43.588 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0: {} 2021-01-15T19:10:43.609 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:43.621 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:43.620+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:43.629 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:43.644 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:43.645 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:43.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:43.646 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:43.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.646 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:666 3.cs0 3:3e4f5332:::smithi00140323-33:head [copy-from ver 38] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:43.646 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:43.646 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 110.387594842, 2021-01-15T19:10:43.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001, 2021-01-15T19:10:43.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.647 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.648 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 666 2021-01-15T19:10:43.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.648 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:43.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253262+0000", 2021-01-15T19:10:43.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2819999999999998e-05 2021-01-15T19:10:43.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253325+0000", 2021-01-15T19:10:43.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2398999999999999e-05 2021-01-15T19:10:43.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253337+0000", 2021-01-15T19:10:43.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1183e-05 2021-01-15T19:10:43.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253358+0000", 2021-01-15T19:10:43.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0879e-05 2021-01-15T19:10:43.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253369+0000", 2021-01-15T19:10:43.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2187000000000005e-05 2021-01-15T19:10:43.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253441+0000", 2021-01-15T19:10:43.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000117964 2021-01-15T19:10:43.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253559+0000", 2021-01-15T19:10:43.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011791625999999999 2021-01-15T19:10:43.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.265351+0000", 2021-01-15T19:10:43.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011424376 2021-01-15T19:10:43.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276775+0000", 2021-01-15T19:10:43.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2426000000000001e-05 2021-01-15T19:10:43.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276848+0000", 2021-01-15T19:10:43.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002434318 2021-01-15T19:10:43.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279282+0000", 2021-01-15T19:10:43.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026020178000000001 2021-01-15T19:10:43.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.663 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.663 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:667 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:43.664 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:43.664 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 110.387429729, 2021-01-15T19:10:43.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999, 2021-01-15T19:10:43.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.665 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 667 2021-01-15T19:10:43.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.666 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:43.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253427+0000", 2021-01-15T19:10:43.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9338000000000002e-05 2021-01-15T19:10:43.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253466+0000", 2021-01-15T19:10:43.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.0119999999999994e-06 2021-01-15T19:10:43.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253475+0000", 2021-01-15T19:10:43.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4296e-05 2021-01-15T19:10:43.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253490+0000", 2021-01-15T19:10:43.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5199999999999999e-06 2021-01-15T19:10:43.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253494+0000", 2021-01-15T19:10:43.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026853100000000002 2021-01-15T19:10:43.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253763+0000", 2021-01-15T19:10:43.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.4134999999999998e-05 2021-01-15T19:10:43.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:43.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.253797+0000", 2021-01-15T19:10:43.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022170481999999998 2021-01-15T19:10:43.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.275967+0000", 2021-01-15T19:10:43.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.8261e-05 2021-01-15T19:10:43.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.276065+0000", 2021-01-15T19:10:43.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032746580000000002 2021-01-15T19:10:43.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279340+0000", 2021-01-15T19:10:43.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000112039 2021-01-15T19:10:43.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279452+0000", 2021-01-15T19:10:43.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151352 2021-01-15T19:10:43.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.279604+0000", 2021-01-15T19:10:43.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026176623999999999 2021-01-15T19:10:43.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:724 3.cs0 3:3e4f5332:::smithi00140323-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e71)", 2021-01-15T19:10:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 104.38002304699999, 2021-01-15T19:10:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001, 2021-01-15T19:10:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 724 2021-01-15T19:10:43.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.684 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:43.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260834+0000", 2021-01-15T19:10:43.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4536999999999997e-05 2021-01-15T19:10:43.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260878+0000", 2021-01-15T19:10:43.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4229999999999999e-06 2021-01-15T19:10:43.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260886+0000", 2021-01-15T19:10:43.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2906e-05 2021-01-15T19:10:43.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260898+0000", 2021-01-15T19:10:43.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2920000000000003e-06 2021-01-15T19:10:43.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.260904+0000", 2021-01-15T19:10:43.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032811400000000001 2021-01-15T19:10:43.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264185+0000", 2021-01-15T19:10:43.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3961999999999997e-05 2021-01-15T19:10:43.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:43.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.264229+0000", 2021-01-15T19:10:43.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015962311 2021-01-15T19:10:43.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280191+0000", 2021-01-15T19:10:43.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00019350899999999999 2021-01-15T19:10:43.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.280385+0000", 2021-01-15T19:10:43.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0063301090000000004 2021-01-15T19:10:43.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286715+0000", 2021-01-15T19:10:43.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014891500000000001 2021-01-15T19:10:43.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.286864+0000", 2021-01-15T19:10:43.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158591 2021-01-15T19:10:43.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:59.287022+0000", 2021-01-15T19:10:43.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026188695000000001 2021-01-15T19:10:43.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.697 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:814 3.fs0 3:f9b5bb77:::smithi00140323-36:head [copy-from ver 38] snapc 36=[36,34,2f,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e79)", 2021-01-15T19:10:43.698 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:43.698 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 96.297286081999999, 2021-01-15T19:10:43.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002, 2021-01-15T19:10:43.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.699 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 814 2021-01-15T19:10:43.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.700 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:43.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343571+0000", 2021-01-15T19:10:43.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0905e-05 2021-01-15T19:10:43.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343641+0000", 2021-01-15T19:10:43.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8400000000000001e-06 2021-01-15T19:10:43.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343650+0000", 2021-01-15T19:10:43.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8247e-05 2021-01-15T19:10:43.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343669+0000", 2021-01-15T19:10:43.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4809999999999997e-06 2021-01-15T19:10:43.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343674+0000", 2021-01-15T19:10:43.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2599000000000002e-05 2021-01-15T19:10:43.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343727+0000", 2021-01-15T19:10:43.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109781 2021-01-15T19:10:43.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343836+0000", 2021-01-15T19:10:43.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011781244999999999 2021-01-15T19:10:43.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.355618+0000", 2021-01-15T19:10:43.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014217357 2021-01-15T19:10:43.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369835+0000", 2021-01-15T19:10:43.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000113192 2021-01-15T19:10:43.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369948+0000", 2021-01-15T19:10:43.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001051169 2021-01-15T19:10:43.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.370999+0000", 2021-01-15T19:10:43.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027428816000000002 2021-01-15T19:10:43.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.713 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:815 3.fs0 3:f9b5bb77:::smithi00140323-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e79)", 2021-01-15T19:10:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 96.297136468000005, 2021-01-15T19:10:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807, 2021-01-15T19:10:43.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 815 2021-01-15T19:10:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:43.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343720+0000", 2021-01-15T19:10:43.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0120999999999998e-05 2021-01-15T19:10:43.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343760+0000", 2021-01-15T19:10:43.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6279999999999994e-06 2021-01-15T19:10:43.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343769+0000", 2021-01-15T19:10:43.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6197000000000001e-05 2021-01-15T19:10:43.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343785+0000", 2021-01-15T19:10:43.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3310000000000003e-06 2021-01-15T19:10:43.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343790+0000", 2021-01-15T19:10:43.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000188622 2021-01-15T19:10:43.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.343979+0000", 2021-01-15T19:10:43.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8069000000000001e-05 2021-01-15T19:10:43.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:43.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.344057+0000", 2021-01-15T19:10:43.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025495675999999998 2021-01-15T19:10:43.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369553+0000", 2021-01-15T19:10:43.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000151008 2021-01-15T19:10:43.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.369704+0000", 2021-01-15T19:10:43.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001333049 2021-01-15T19:10:43.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371037+0000", 2021-01-15T19:10:43.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4074999999999999e-05 2021-01-15T19:10:43.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371101+0000", 2021-01-15T19:10:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3030999999999998e-05 2021-01-15T19:10:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:07.371184+0000", 2021-01-15T19:10:43.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027463807 2021-01-15T19:10:43.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.733 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:863 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~720896] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:43.733 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:43.733 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.276838785999999, 2021-01-15T19:10:43.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999, 2021-01-15T19:10:43.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.734 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 863 2021-01-15T19:10:43.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.735 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:43.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364018+0000", 2021-01-15T19:10:43.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036319899999999998 2021-01-15T19:10:43.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364381+0000", 2021-01-15T19:10:43.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0137e-05 2021-01-15T19:10:43.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364391+0000", 2021-01-15T19:10:43.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9522999999999999e-05 2021-01-15T19:10:43.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364411+0000", 2021-01-15T19:10:43.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6609999999999995e-06 2021-01-15T19:10:43.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364420+0000", 2021-01-15T19:10:43.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2737e-05 2021-01-15T19:10:43.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364473+0000", 2021-01-15T19:10:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.58e-06 2021-01-15T19:10:43.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.364477+0000", 2021-01-15T19:10:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.004732197 2021-01-15T19:10:43.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369209+0000", 2021-01-15T19:10:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031182900000000001 2021-01-15T19:10:43.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369521+0000", 2021-01-15T19:10:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014171170000000001 2021-01-15T19:10:43.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370938+0000", 2021-01-15T19:10:43.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090093269999999993 2021-01-15T19:10:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379947+0000", 2021-01-15T19:10:43.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034319367000000003 2021-01-15T19:10:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414266+0000", 2021-01-15T19:10:43.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037266799999999999 2021-01-15T19:10:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414639+0000", 2021-01-15T19:10:43.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0506213419999999 2021-01-15T19:10:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:864 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~540672] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.273782135000005, 2021-01-15T19:10:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941, 2021-01-15T19:10:43.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 864 2021-01-15T19:10:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367074+0000", 2021-01-15T19:10:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025175799999999998 2021-01-15T19:10:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367326+0000", 2021-01-15T19:10:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0151e-05 2021-01-15T19:10:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367336+0000", 2021-01-15T19:10:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7081000000000001e-05 2021-01-15T19:10:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367353+0000", 2021-01-15T19:10:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7129999999999994e-06 2021-01-15T19:10:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367362+0000", 2021-01-15T19:10:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9826000000000005e-05 2021-01-15T19:10:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367432+0000", 2021-01-15T19:10:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.7390000000000001e-06 2021-01-15T19:10:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.367436+0000", 2021-01-15T19:10:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.01409762 2021-01-15T19:10:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381533+0000", 2021-01-15T19:10:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00024178700000000001 2021-01-15T19:10:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.381775+0000", 2021-01-15T19:10:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012628120000000001 2021-01-15T19:10:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.383038+0000", 2021-01-15T19:10:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030863550999999999 2021-01-15T19:10:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413902+0000", 2021-01-15T19:10:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001065147 2021-01-15T19:10:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414967+0000", 2021-01-15T19:10:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042175600000000002 2021-01-15T19:10:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415388+0000", 2021-01-15T19:10:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.048313941 2021-01-15T19:10:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:865 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~557056] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:43.768 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.270768872999994, 2021-01-15T19:10:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001, 2021-01-15T19:10:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 865 2021-01-15T19:10:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.770 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370088+0000", 2021-01-15T19:10:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023701000000000001 2021-01-15T19:10:43.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370325+0000", 2021-01-15T19:10:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0929999999999999e-06 2021-01-15T19:10:43.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370331+0000", 2021-01-15T19:10:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0811e-05 2021-01-15T19:10:43.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370342+0000", 2021-01-15T19:10:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.2990000000000002e-06 2021-01-15T19:10:43.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370348+0000", 2021-01-15T19:10:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0600999999999999e-05 2021-01-15T19:10:43.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370399+0000", 2021-01-15T19:10:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.818e-06 2021-01-15T19:10:43.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.370401+0000", 2021-01-15T19:10:43.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.015073474 2021-01-15T19:10:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385475+0000", 2021-01-15T19:10:43.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000265087 2021-01-15T19:10:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.385740+0000", 2021-01-15T19:10:43.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001442512 2021-01-15T19:10:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.387182+0000", 2021-01-15T19:10:43.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028225726 2021-01-15T19:10:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415408+0000", 2021-01-15T19:10:43.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025971799999999998 2021-01-15T19:10:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415668+0000", 2021-01-15T19:10:43.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025643699999999998 2021-01-15T19:10:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.415924+0000", 2021-01-15T19:10:43.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0458365860000001 2021-01-15T19:10:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:866 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~507904] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.268021262000005, 2021-01-15T19:10:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999, 2021-01-15T19:10:43.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 866 2021-01-15T19:10:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.372835+0000", 2021-01-15T19:10:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00034269600000000003 2021-01-15T19:10:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373178+0000", 2021-01-15T19:10:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2109999999999999e-05 2021-01-15T19:10:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373190+0000", 2021-01-15T19:10:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.003e-05 2021-01-15T19:10:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373210+0000", 2021-01-15T19:10:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.9680000000000001e-06 2021-01-15T19:10:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373220+0000", 2021-01-15T19:10:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8279999999999998e-05 2021-01-15T19:10:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373278+0000", 2021-01-15T19:10:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9730000000000002e-06 2021-01-15T19:10:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373281+0000", 2021-01-15T19:10:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.042761437 2021-01-15T19:10:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416043+0000", 2021-01-15T19:10:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011653400000000001 2021-01-15T19:10:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416159+0000", 2021-01-15T19:10:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072145499999999997 2021-01-15T19:10:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.416881+0000", 2021-01-15T19:10:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0064967499999999999 2021-01-15T19:10:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423378+0000", 2021-01-15T19:10:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015534300000000001 2021-01-15T19:10:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423533+0000", 2021-01-15T19:10:43.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030109000000000001 2021-01-15T19:10:43.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423834+0000", 2021-01-15T19:10:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0509986659999999 2021-01-15T19:10:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:867 3.cs0 3:3e4f5332:::smithi00140323-33:head [setxattr _header (57) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:43.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.26758744, 2021-01-15T19:10:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828, 2021-01-15T19:10:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 867 2021-01-15T19:10:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373269+0000", 2021-01-15T19:10:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5634000000000003e-05 2021-01-15T19:10:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373325+0000", 2021-01-15T19:10:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.357e-06 2021-01-15T19:10:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373332+0000", 2021-01-15T19:10:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5715000000000001e-05 2021-01-15T19:10:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373348+0000", 2021-01-15T19:10:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2469999999999999e-06 2021-01-15T19:10:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373353+0000", 2021-01-15T19:10:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0881999999999999e-05 2021-01-15T19:10:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373394+0000", 2021-01-15T19:10:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0109999999999999e-06 2021-01-15T19:10:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373395+0000", 2021-01-15T19:10:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.046902019 2021-01-15T19:10:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420297+0000", 2021-01-15T19:10:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021355199999999999 2021-01-15T19:10:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.420511+0000", 2021-01-15T19:10:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00072261999999999999 2021-01-15T19:10:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.421233+0000", 2021-01-15T19:10:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0043733330000000001 2021-01-15T19:10:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425607+0000", 2021-01-15T19:10:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.1914000000000001e-05 2021-01-15T19:10:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425698+0000", 2021-01-15T19:10:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00063954400000000003 2021-01-15T19:10:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426338+0000", 2021-01-15T19:10:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.053068828 2021-01-15T19:10:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:868 3.cs0 3:3e4f5332:::smithi00140323-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.267463281000005, 2021-01-15T19:10:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302, 2021-01-15T19:10:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 868 2021-01-15T19:10:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373393+0000", 2021-01-15T19:10:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9184000000000001e-05 2021-01-15T19:10:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373413+0000", 2021-01-15T19:10:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4859999999999997e-06 2021-01-15T19:10:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373418+0000", 2021-01-15T19:10:43.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3459999999999996e-06 2021-01-15T19:10:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373425+0000", 2021-01-15T19:10:43.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.9050000000000001e-06 2021-01-15T19:10:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373428+0000", 2021-01-15T19:10:43.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2072e-05 2021-01-15T19:10:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373470+0000", 2021-01-15T19:10:43.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6670000000000001e-06 2021-01-15T19:10:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.373472+0000", 2021-01-15T19:10:43.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0504642740000001 2021-01-15T19:10:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.423936+0000", 2021-01-15T19:10:43.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000220137 2021-01-15T19:10:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.424156+0000", 2021-01-15T19:10:43.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002233287 2021-01-15T19:10:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426390+0000", 2021-01-15T19:10:43.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120664 2021-01-15T19:10:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.426510+0000", 2021-01-15T19:10:43.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018192799999999999 2021-01-15T19:10:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.428330+0000", 2021-01-15T19:10:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.054936302 2021-01-15T19:10:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:876 3.14s0 3:2d79b238:::smithi00140323-40:head [stat,delete] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 91.162786745999995, 2021-01-15T19:10:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002, 2021-01-15T19:10:43.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 876 2021-01-15T19:10:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478070+0000", 2021-01-15T19:10:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.5238000000000001e-05 2021-01-15T19:10:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478095+0000", 2021-01-15T19:10:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8200000000000003e-06 2021-01-15T19:10:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478104+0000", 2021-01-15T19:10:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6178999999999999e-05 2021-01-15T19:10:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478120+0000", 2021-01-15T19:10:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.867e-06 2021-01-15T19:10:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478129+0000", 2021-01-15T19:10:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1699000000000005e-05 2021-01-15T19:10:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478211+0000", 2021-01-15T19:10:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5829999999999997e-06 2021-01-15T19:10:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.478216+0000", 2021-01-15T19:10:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.876371444 2021-01-15T19:10:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.354588+0000", 2021-01-15T19:10:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000804783 2021-01-15T19:10:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.355392+0000", 2021-01-15T19:10:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.070285142999999994 2021-01-15T19:10:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425678+0000", 2021-01-15T19:10:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014788099999999999 2021-01-15T19:10:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.425826+0000", 2021-01-15T19:10:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00380504 2021-01-15T19:10:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429631+0000", 2021-01-15T19:10:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116982 2021-01-15T19:10:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.429748+0000", 2021-01-15T19:10:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00092146199999999995 2021-01-15T19:10:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.430669+0000", 2021-01-15T19:10:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049054750000000003 2021-01-15T19:10:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.435574+0000", 2021-01-15T19:10:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033847119999999998 2021-01-15T19:10:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.438959+0000", 2021-01-15T19:10:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036306400000000002 2021-01-15T19:10:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.439322+0000", 2021-01-15T19:10:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96125237200000002 2021-01-15T19:10:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.857 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 85/84 ECSubWriteReply(tid=257, last_complete=85'126, committed=1, applied=1))", 2021-01-15T19:10:43.858 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.252847509999995, 2021-01-15T19:10:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666, 2021-01-15T19:10:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:43.859 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388009+0000", 2021-01-15T19:10:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3158000000000002e-05 2021-01-15T19:10:43.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388062+0000", 2021-01-15T19:10:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7630000000000006e-06 2021-01-15T19:10:43.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388071+0000", 2021-01-15T19:10:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4067999999999999e-05 2021-01-15T19:10:43.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388115+0000", 2021-01-15T19:10:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3949999999999998e-06 2021-01-15T19:10:43.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.388121+0000", 2021-01-15T19:10:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025820189 2021-01-15T19:10:43.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.413942+0000", 2021-01-15T19:10:43.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00071509299999999998 2021-01-15T19:10:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.414657+0000", 2021-01-15T19:10:43.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026647666 2021-01-15T19:10:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1179 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e124)", 2021-01-15T19:10:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.410126609000002, 2021-01-15T19:10:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217, 2021-01-15T19:10:43.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1179 2021-01-15T19:10:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230730+0000", 2021-01-15T19:10:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9990000000000002e-05 2021-01-15T19:10:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230770+0000", 2021-01-15T19:10:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5860000000000005e-06 2021-01-15T19:10:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230779+0000", 2021-01-15T19:10:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3725e-05 2021-01-15T19:10:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230792+0000", 2021-01-15T19:10:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9169999999999997e-06 2021-01-15T19:10:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.230797+0000", 2021-01-15T19:10:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012361906000000001 2021-01-15T19:10:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243159+0000", 2021-01-15T19:10:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013923200000000001 2021-01-15T19:10:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.243298+0000", 2021-01-15T19:10:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010443971999999999 2021-01-15T19:10:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253742+0000", 2021-01-15T19:10:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000143287 2021-01-15T19:10:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.253886+0000", 2021-01-15T19:10:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001017602 2021-01-15T19:10:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:52.254903+0000", 2021-01-15T19:10:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024173217 2021-01-15T19:10:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1381 3.cs0 3:3ef45b66:::smithi00140323-11:head [copy-from ver 50] snapc 71=[71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e142)", 2021-01-15T19:10:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:43.883 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.435547145000001, 2021-01-15T19:10:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076, 2021-01-15T19:10:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1381 2021-01-15T19:10:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.885 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205309+0000", 2021-01-15T19:10:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6891999999999998e-05 2021-01-15T19:10:43.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205356+0000", 2021-01-15T19:10:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4290000000000001e-06 2021-01-15T19:10:43.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205365+0000", 2021-01-15T19:10:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5362999999999998e-05 2021-01-15T19:10:43.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205380+0000", 2021-01-15T19:10:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1709999999999996e-06 2021-01-15T19:10:43.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205387+0000", 2021-01-15T19:10:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0346999999999996e-05 2021-01-15T19:10:43.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205458+0000", 2021-01-15T19:10:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0080000000000002e-06 2021-01-15T19:10:43.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205466+0000", 2021-01-15T19:10:43.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027937019999999999 2021-01-15T19:10:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208259+0000", 2021-01-15T19:10:43.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000137386 2021-01-15T19:10:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208397+0000", 2021-01-15T19:10:43.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087910060000000005 2021-01-15T19:10:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.217188+0000", 2021-01-15T19:10:43.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0099349 2021-01-15T19:10:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.227123+0000", 2021-01-15T19:10:43.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030732759999999998 2021-01-15T19:10:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230196+0000", 2021-01-15T19:10:43.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015259599999999999 2021-01-15T19:10:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.230349+0000", 2021-01-15T19:10:43.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025039076 2021-01-15T19:10:43.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.900 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.901 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1382 3.cs0 3:3ef45b66:::smithi00140323-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e142)", 2021-01-15T19:10:43.901 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:43.901 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 33.435415484000004, 2021-01-15T19:10:43.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002, 2021-01-15T19:10:43.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.902 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1382 2021-01-15T19:10:43.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.903 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:43.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205441+0000", 2021-01-15T19:10:43.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9499000000000001e-05 2021-01-15T19:10:43.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205461+0000", 2021-01-15T19:10:43.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0709999999999996e-06 2021-01-15T19:10:43.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205465+0000", 2021-01-15T19:10:43.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.456e-06 2021-01-15T19:10:43.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205473+0000", 2021-01-15T19:10:43.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.101e-06 2021-01-15T19:10:43.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205476+0000", 2021-01-15T19:10:43.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1783000000000002e-05 2021-01-15T19:10:43.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205528+0000", 2021-01-15T19:10:43.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0718e-05 2021-01-15T19:10:43.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:43.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.205539+0000", 2021-01-15T19:10:43.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0030667680000000001 2021-01-15T19:10:43.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208606+0000", 2021-01-15T19:10:43.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1095999999999999e-05 2021-01-15T19:10:43.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:43.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.208667+0000", 2021-01-15T19:10:43.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017719619999999998 2021-01-15T19:10:43.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226386+0000", 2021-01-15T19:10:43.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000154279 2021-01-15T19:10:43.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:43.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.226541+0000", 2021-01-15T19:10:43.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0051398169999999996 2021-01-15T19:10:43.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231680+0000", 2021-01-15T19:10:43.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012625000000000001 2021-01-15T19:10:43.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231807+0000", 2021-01-15T19:10:43.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000153857 2021-01-15T19:10:43.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:10.231960+0000", 2021-01-15T19:10:43.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026519315000000002 2021-01-15T19:10:43.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1672 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~6144000,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e167)", 2021-01-15T19:10:43.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.513472+0000", 2021-01-15T19:10:43.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1273844900000007, 2021-01-15T19:10:43.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023736568999999999, 2021-01-15T19:10:43.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1672 2021-01-15T19:10:43.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513472+0000", 2021-01-15T19:10:43.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513472+0000", 2021-01-15T19:10:43.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6097000000000002e-05 2021-01-15T19:10:43.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513528+0000", 2021-01-15T19:10:43.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.661e-06 2021-01-15T19:10:43.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513536+0000", 2021-01-15T19:10:43.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3631999999999999e-05 2021-01-15T19:10:43.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513550+0000", 2021-01-15T19:10:43.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1689999999999997e-06 2021-01-15T19:10:43.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513556+0000", 2021-01-15T19:10:43.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0014363049999999999 2021-01-15T19:10:43.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.514992+0000", 2021-01-15T19:10:43.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1141000000000001e-05 2021-01-15T19:10:43.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.515073+0000", 2021-01-15T19:10:43.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022135564 2021-01-15T19:10:43.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.537209+0000", 2021-01-15T19:10:43.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023736568999999999 2021-01-15T19:10:43.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.931 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.931 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1673 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~6144000,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e167)", 2021-01-15T19:10:43.932 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.513609+0000", 2021-01-15T19:10:43.932 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1272479539999996, 2021-01-15T19:10:43.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028079653999999999, 2021-01-15T19:10:43.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.933 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1673 2021-01-15T19:10:43.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.934 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.934 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513609+0000", 2021-01-15T19:10:43.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513609+0000", 2021-01-15T19:10:43.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6870999999999998e-05 2021-01-15T19:10:43.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513666+0000", 2021-01-15T19:10:43.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5220000000000004e-06 2021-01-15T19:10:43.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513672+0000", 2021-01-15T19:10:43.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3616e-05 2021-01-15T19:10:43.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513686+0000", 2021-01-15T19:10:43.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0019999999999997e-06 2021-01-15T19:10:43.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513691+0000", 2021-01-15T19:10:43.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001583112 2021-01-15T19:10:43.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.515274+0000", 2021-01-15T19:10:43.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000121321 2021-01-15T19:10:43.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.515395+0000", 2021-01-15T19:10:43.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026293210000000001 2021-01-15T19:10:43.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.541688+0000", 2021-01-15T19:10:43.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.028079653999999999 2021-01-15T19:10:43.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.943 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.944 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1674 3.cs0 3:3f312ec7:::smithi00140323-7:head [read 0~6144000,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e167)", 2021-01-15T19:10:43.944 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.513732+0000", 2021-01-15T19:10:43.944 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1271250409999993, 2021-01-15T19:10:43.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032831451999999997, 2021-01-15T19:10:43.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:43.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.945 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1674 2021-01-15T19:10:43.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.946 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513732+0000", 2021-01-15T19:10:43.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513732+0000", 2021-01-15T19:10:43.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2164999999999999e-05 2021-01-15T19:10:43.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513784+0000", 2021-01-15T19:10:43.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-06 2021-01-15T19:10:43.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513790+0000", 2021-01-15T19:10:43.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1747e-05 2021-01-15T19:10:43.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.951 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513802+0000", 2021-01-15T19:10:43.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5190000000000003e-06 2021-01-15T19:10:43.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.513806+0000", 2021-01-15T19:10:43.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050619259999999996 2021-01-15T19:10:43.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.518868+0000", 2021-01-15T19:10:43.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114505 2021-01-15T19:10:43.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.518983+0000", 2021-01-15T19:10:43.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027580470999999999 2021-01-15T19:10:43.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.546563+0000", 2021-01-15T19:10:43.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032831451999999997 2021-01-15T19:10:43.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.956 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1676 3.cs0 3:3e4f5332:::smithi00140323-33:head [append 0~458752] snapc 86=[86,84,83,82,7d,7b,7a,72,70,6f,6b,67,64] ondisk+write+known_if_redirected e167)", 2021-01-15T19:10:43.956 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:35.527635+0000", 2021-01-15T19:10:43.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 8.1132218920000003, 2021-01-15T19:10:43.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026863965, 2021-01-15T19:10:43.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:43.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:43.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:43.957 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:43.958 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:43.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1676 2021-01-15T19:10:43.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.958 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:43.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:43.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527635+0000", 2021-01-15T19:10:43.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:43.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527635+0000", 2021-01-15T19:10:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026864400000000001 2021-01-15T19:10:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527903+0000", 2021-01-15T19:10:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0713000000000001e-05 2021-01-15T19:10:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527914+0000", 2021-01-15T19:10:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.504e-05 2021-01-15T19:10:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:43.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527929+0000", 2021-01-15T19:10:43.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5430000000000001e-06 2021-01-15T19:10:43.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:43.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.527935+0000", 2021-01-15T19:10:43.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018745034000000001 2021-01-15T19:10:43.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:43.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.546680+0000", 2021-01-15T19:10:43.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3111999999999999e-05 2021-01-15T19:10:43.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:43.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.546773+0000", 2021-01-15T19:10:43.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00060443699999999999 2021-01-15T19:10:43.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:43.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.547377+0000", 2021-01-15T19:10:43.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006880387 2021-01-15T19:10:43.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:43.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.554258+0000", 2021-01-15T19:10:43.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000145993 2021-01-15T19:10:43.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:43.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.554404+0000", 2021-01-15T19:10:43.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5062000000000006e-05 2021-01-15T19:10:43.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:43.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:43.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:43.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:35.554499+0000", 2021-01-15T19:10:43.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026863965 2021-01-15T19:10:43.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.971 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:43.971 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:43.971 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:43.973 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:43.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:43.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:43.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:43.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:43.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.976 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:43.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:43.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.977 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:43.977 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:43.978 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:43.978 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:43.972+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:44.074 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:44.073+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:44.146 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:44.147 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:44.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:44.175 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:44.174+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:44.260 INFO:tasks.rados.rados.0.smithi001.stdout:747: done (2 left) 2021-01-15T19:10:44.260 INFO:tasks.rados.rados.0.smithi001.stdout:748: done (1 left) 2021-01-15T19:10:44.260 INFO:tasks.rados.rados.0.smithi001.stdout:749: done (0 left) 2021-01-15T19:10:44.260 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 43 current snap is 80 2021-01-15T19:10:44.261 INFO:tasks.rados.rados.0.smithi001.stdout:750: seq_num 262 ranges {5406720=794624,6201344=589824,6791168=614400} 2021-01-15T19:10:44.275 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:44.274+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:44.293 INFO:tasks.rados.rados.0.smithi001.stdout:750: writing smithi00140323-43 from 5406720 to 6201344 tid 1 2021-01-15T19:10:44.294 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:44.295 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:44.296 INFO:tasks.rados.rados.0.smithi001.stdout:750: writing smithi00140323-43 from 6201344 to 6791168 tid 2 2021-01-15T19:10:44.300 INFO:tasks.rados.rados.0.smithi001.stdout:750: writing smithi00140323-43 from 6791168 to 7405568 tid 3 2021-01-15T19:10:44.300 INFO:tasks.rados.rados.0.smithi001.stdout:751: setattr oid 12 current snap is 80 2021-01-15T19:10:44.301 INFO:tasks.rados.rados.0.smithi001.stdout:752: delete oid 33 current snap is 80 2021-01-15T19:10:44.306 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 12 v 150 (ObjNum 179408768 snap 22026 seq_num 0) dirty exists 2021-01-15T19:10:44.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:44.309 INFO:tasks.rados.rados.0.smithi001.stdout:751: done (2 left) 2021-01-15T19:10:44.309 INFO:tasks.rados.rados.0.smithi001.stdout:752: done (1 left) 2021-01-15T19:10:44.309 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 38 current snap is 80 2021-01-15T19:10:44.310 INFO:tasks.rados.rados.0.smithi001.stdout:753: seq_num 264 ranges {6889472=425984,7315456=761856,8077312=770048,8847360=73728} 2021-01-15T19:10:44.347 INFO:tasks.rados.rados.0.smithi001.stdout:753: writing smithi00140323-38 from 6889472 to 7315456 tid 1 2021-01-15T19:10:44.351 INFO:tasks.rados.rados.0.smithi001.stdout:753: writing smithi00140323-38 from 7315456 to 8077312 tid 2 2021-01-15T19:10:44.356 INFO:tasks.rados.rados.0.smithi001.stdout:753: writing smithi00140323-38 from 8077312 to 8847360 tid 3 2021-01-15T19:10:44.357 INFO:tasks.rados.rados.0.smithi001.stdout:753: writing smithi00140323-38 from 8847360 to 8921088 tid 4 2021-01-15T19:10:44.357 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 7 current snap is 80 2021-01-15T19:10:44.357 INFO:tasks.rados.rados.0.smithi001.stdout:754: seq_num 265 ranges {6144000=442368,6586368=647168,7233536=573440,7806976=385024} 2021-01-15T19:10:44.376 INFO:tasks.ceph.osd.0.smithi001.stderr:2021-01-15T19:10:44.375+0000 7fb6ce115700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 34820) UID: 0 2021-01-15T19:10:44.392 INFO:tasks.rados.rados.0.smithi001.stdout:754: writing smithi00140323-7 from 6144000 to 6586368 tid 1 2021-01-15T19:10:44.396 INFO:tasks.rados.rados.0.smithi001.stdout:754: writing smithi00140323-7 from 6586368 to 7233536 tid 2 2021-01-15T19:10:44.400 INFO:tasks.rados.rados.0.smithi001.stdout:754: writing smithi00140323-7 from 7233536 to 7806976 tid 3 2021-01-15T19:10:44.403 INFO:tasks.rados.rados.0.smithi001.stdout:754: writing smithi00140323-7 from 7806976 to 8192000 tid 4 2021-01-15T19:10:44.403 INFO:tasks.rados.rados.0.smithi001.stdout:755: rollback oid 29 current snap is 80 2021-01-15T19:10:44.404 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 29 to 57 2021-01-15T19:10:44.404 INFO:tasks.rados.rados.0.smithi001.stdout:750: finishing write tid 1 to smithi00140323-43 2021-01-15T19:10:44.404 INFO:tasks.rados.rados.0.smithi001.stdout:750: finishing write tid 2 to smithi00140323-43 2021-01-15T19:10:44.404 INFO:tasks.rados.rados.0.smithi001.stdout:750: finishing write tid 3 to smithi00140323-43 2021-01-15T19:10:44.405 INFO:tasks.rados.rados.0.smithi001.stdout:750: finishing write tid 4 to smithi00140323-43 2021-01-15T19:10:44.406 INFO:tasks.rados.rados.0.smithi001.stdout:750: finishing write tid 5 to smithi00140323-43 2021-01-15T19:10:44.406 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 43 v 368 (ObjNum 262 snap 80 seq_num 262) dirty exists 2021-01-15T19:10:44.406 INFO:tasks.rados.rados.0.smithi001.stdout:750: left oid 43 (ObjNum 262 snap 80 seq_num 262) 2021-01-15T19:10:44.406 INFO:tasks.rados.rados.0.smithi001.stdout:753: finishing write tid 1 to smithi00140323-38 2021-01-15T19:10:44.407 INFO:tasks.rados.rados.0.smithi001.stdout:753: finishing write tid 2 to smithi00140323-38 2021-01-15T19:10:44.407 INFO:tasks.rados.rados.0.smithi001.stdout:753: finishing write tid 3 to smithi00140323-38 2021-01-15T19:10:44.407 INFO:tasks.rados.rados.0.smithi001.stdout:753: finishing write tid 4 to smithi00140323-38 2021-01-15T19:10:44.407 INFO:tasks.rados.rados.0.smithi001.stdout:750: done (3 left) 2021-01-15T19:10:44.408 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 34 current snap is 80 2021-01-15T19:10:44.408 INFO:tasks.rados.rados.0.smithi001.stdout:753: finishing write tid 5 to smithi00140323-38 2021-01-15T19:10:44.408 INFO:tasks.rados.rados.0.smithi001.stdout:756: seq_num 266 ranges {0=458752,458752=524288,983040=729088,1712128=352256} 2021-01-15T19:10:44.409 INFO:tasks.rados.rados.0.smithi001.stdout:756: writing smithi00140323-34 from 0 to 458752 tid 1 2021-01-15T19:10:44.409 INFO:tasks.rados.rados.0.smithi001.stdout:756: writing smithi00140323-34 from 458752 to 983040 tid 2 2021-01-15T19:10:44.414 INFO:tasks.rados.rados.0.smithi001.stdout:756: writing smithi00140323-34 from 983040 to 1712128 tid 3 2021-01-15T19:10:44.417 INFO:tasks.rados.rados.0.smithi001.stdout:756: writing smithi00140323-34 from 1712128 to 2064384 tid 4 2021-01-15T19:10:44.417 INFO:tasks.rados.rados.0.smithi001.stdout:753: finishing write tid 6 to smithi00140323-38 2021-01-15T19:10:44.418 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 38 v 70 (ObjNum 264 snap 80 seq_num 264) dirty exists 2021-01-15T19:10:44.418 INFO:tasks.rados.rados.0.smithi001.stdout:753: left oid 38 (ObjNum 264 snap 80 seq_num 264) 2021-01-15T19:10:44.418 INFO:tasks.rados.rados.0.smithi001.stdout:754: finishing write tid 1 to smithi00140323-7 2021-01-15T19:10:44.418 INFO:tasks.rados.rados.0.smithi001.stdout:754: finishing write tid 2 to smithi00140323-7 2021-01-15T19:10:44.418 INFO:tasks.rados.rados.0.smithi001.stdout:754: finishing write tid 3 to smithi00140323-7 2021-01-15T19:10:44.419 INFO:tasks.rados.rados.0.smithi001.stdout:755: finishing rollback tid 1 to smithi00140323-29 2021-01-15T19:10:44.419 INFO:tasks.rados.rados.0.smithi001.stdout:755: finishing rollback tid 2 to smithi00140323-29 2021-01-15T19:10:44.419 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 29 v 199 (ObjNum 177 snap 56 seq_num 177) dirty exists 2021-01-15T19:10:44.420 INFO:tasks.rados.rados.0.smithi001.stdout:754: finishing write tid 4 to smithi00140323-7 2021-01-15T19:10:44.421 INFO:tasks.rados.rados.0.smithi001.stdout:753: done (3 left) 2021-01-15T19:10:44.421 INFO:tasks.rados.rados.0.smithi001.stdout:755: done (2 left) 2021-01-15T19:10:44.421 INFO:tasks.rados.rados.0.smithi001.stdout:757: rmattr oid 47 current snap is 80 2021-01-15T19:10:44.421 INFO:tasks.rados.rados.0.smithi001.stdout:757: done (2 left) 2021-01-15T19:10:44.422 INFO:tasks.rados.rados.0.smithi001.stdout:758: snap_remove snap 70 2021-01-15T19:10:44.477 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:44.476+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:44.497 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.05886884900001, 2021-01-15T19:10:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:10:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:10:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:10:44.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:10:44.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:10:44.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:10:44.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:10:44.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:10:44.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:44.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:10:44.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:10:44.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:10:44.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:10:44.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:10:44.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:10:44.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:10:44.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:10:44.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:10:44.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:10:44.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:10:44.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:10:44.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:10:44.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:10:44.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:10:44.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:10:44.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:10:44.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:10:44.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:10:44.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:10:44.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:10:44.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:10:44.538 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.539 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.539 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:44.540 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:44.540 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.05872954900001, 2021-01-15T19:10:44.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:10:44.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.542 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:10:44.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.543 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:44.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:44.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:10:44.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:10:44.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:44.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:10:44.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:10:44.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:10:44.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:10:44.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:10:44.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:10:44.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:10:44.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:10:44.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:10:44.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:10:44.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:10:44.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:10:44.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:10:44.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:10:44.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:10:44.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:10:44.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:10:44.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:10:44.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:10:44.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:10:44.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:10:44.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:10:44.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:10:44.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:10:44.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:10:44.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:10:44.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.566 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.567 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:44.567 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:44.567 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.058608443, 2021-01-15T19:10:44.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:10:44.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.568 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.569 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:10:44.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.569 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:44.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:44.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:10:44.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:10:44.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:10:44.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:10:44.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:10:44.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:10:44.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:10:44.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:10:44.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:10:44.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:10:44.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:10:44.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:10:44.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:10:44.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:10:44.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:10:44.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:10:44.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:10:44.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:10:44.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:10:44.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:10:44.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:10:44.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:10:44.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:10:44.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:10:44.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:10:44.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:10:44.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:10:44.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:10:44.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:10:44.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.588 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:44.588 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:44.588 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 120.288327569, 2021-01-15T19:10:44.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:10:44.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:10:44.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.590 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:44.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:44.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:10:44.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:10:44.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:10:44.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:10:44.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:10:44.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:10:44.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:10:44.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:10:44.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:10:44.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:10:44.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:10:44.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:10:44.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:10:44.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:10:44.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:10:44.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:10:44.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:10:44.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:10:44.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:10:44.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:10:44.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:10:44.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:10:44.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:10:44.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:10:44.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:10:44.604 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.604 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.605 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:44.605 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:44.606 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 120.279349391, 2021-01-15T19:10:44.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:10:44.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.607 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:10:44.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.608 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:44.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:44.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:10:44.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:10:44.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:10:44.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:10:44.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:10:44.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:10:44.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:44.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:10:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:10:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:44.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:10:44.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:10:44.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:10:44.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:10:44.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:10:44.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:10:44.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:10:44.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:10:44.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:10:44.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:10:44.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:10:44.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:10:44.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:10:44.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:10:44.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:10:44.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.622 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.623 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:44.623 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:44.623 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 120.278829691, 2021-01-15T19:10:44.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:10:44.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:10:44.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.625 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:44.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:44.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:10:44.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:10:44.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:10:44.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:10:44.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:10:44.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:10:44.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:10:44.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:10:44.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:10:44.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:10:44.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:10:44.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:10:44.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:10:44.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:10:44.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:10:44.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:10:44.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:10:44.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:10:44.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:10:44.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:10:44.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:10:44.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:10:44.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:10:44.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:10:44.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:10:44.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.640 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:44.641 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:44.641 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 120.277148737, 2021-01-15T19:10:44.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:10:44.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:10:44.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.643 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:44.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:44.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:10:44.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:10:44.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:10:44.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:10:44.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:10:44.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:10:44.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:10:44.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:10:44.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:10:44.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:44.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:10:44.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:10:44.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:10:44.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:10:44.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:10:44.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:10:44.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:10:44.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:10:44.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:10:44.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:10:44.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:10:44.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:10:44.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:10:44.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:10:44.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:10:44.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.657 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.658 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:44.658 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:44.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 120.277015783, 2021-01-15T19:10:44.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:10:44.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:44.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:10:44.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.660 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:44.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:44.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:10:44.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:10:44.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:10:44.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:10:44.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:44.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:10:44.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:10:44.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:10:44.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:10:44.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:10:44.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:10:44.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:44.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:10:44.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:10:44.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:10:44.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:10:44.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:44.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:10:44.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:10:44.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:10:44.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:10:44.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:10:44.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:10:44.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:10:44.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:10:44.673 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.673 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.674 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:44.674 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:44.675 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.314484513, 2021-01-15T19:10:44.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:10:44.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.675 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.676 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:10:44.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.676 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:44.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:44.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:10:44.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:10:44.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:10:44.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:10:44.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:10:44.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:10:44.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:10:44.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:10:44.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:10:44.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:10:44.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:10:44.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:44.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:10:44.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:10:44.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:10:44.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:10:44.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:10:44.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:10:44.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:10:44.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:10:44.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:10:44.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:10:44.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:10:44.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:10:44.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:10:44.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:10:44.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.691 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.691 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.692 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:44.692 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:44.692 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.31431791999999, 2021-01-15T19:10:44.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:10:44.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:44.693 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.693 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.693 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:10:44.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.694 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:44.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:10:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:10:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:10:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:10:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:10:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:10:44.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:10:44.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:10:44.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:10:44.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:10:44.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:10:44.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:44.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:10:44.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:10:44.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:10:44.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:10:44.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:44.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:10:44.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:10:44.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:10:44.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:10:44.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:44.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:10:44.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:10:44.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:10:44.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:10:44.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:10:44.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:10:44.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:10:44.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:10:44.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.710 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:44.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:44.710 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.115779126000007, 2021-01-15T19:10:44.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:10:44.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.712 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:10:44.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.712 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:44.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:44.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:10:44.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:10:44.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:10:44.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:10:44.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:10:44.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:10:44.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:10:44.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:10:44.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:10:44.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:10:44.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:10:44.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:10:44.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:10:44.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:10:44.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:10:44.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:10:44.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:10:44.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:10:44.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:10:44.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:10:44.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:10:44.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:10:44.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:10:44.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:10:44.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:10:44.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.728 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.729 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:44.729 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:44.729 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.113509081000004, 2021-01-15T19:10:44.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:10:44.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.730 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:10:44.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.731 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:44.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:44.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:10:44.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:10:44.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:10:44.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:10:44.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:10:44.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:10:44.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:10:44.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:10:44.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:10:44.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:10:44.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:10:44.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:10:44.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:10:44.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:10:44.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:10:44.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:10:44.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:10:44.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:10:44.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:10:44.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:10:44.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:10:44.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:10:44.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:10:44.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:10:44.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:10:44.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.745 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.746 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:44.746 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:44.746 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.108970889999995, 2021-01-15T19:10:44.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:10:44.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.747 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:10:44.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.748 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:44.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:44.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:10:44.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:10:44.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:10:44.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:10:44.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:10:44.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:10:44.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:44.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:10:44.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:10:44.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:10:44.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:10:44.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:10:44.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:10:44.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:10:44.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:10:44.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:10:44.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:10:44.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:10:44.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:10:44.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:10:44.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:10:44.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:10:44.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:10:44.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:10:44.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:10:44.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.763 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:44.764 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:44.764 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.106385017999997, 2021-01-15T19:10:44.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:10:44.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.765 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:10:44.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.766 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:44.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:44.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:10:44.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:10:44.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:10:44.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:10:44.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:10:44.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:10:44.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:10:44.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:10:44.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:10:44.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:10:44.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:10:44.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:10:44.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:10:44.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:10:44.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:10:44.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:10:44.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:10:44.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:10:44.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:10:44.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:10:44.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:10:44.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:10:44.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:10:44.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:10:44.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:10:44.780 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.780 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.780 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.781 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:44.781 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:44.781 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.106010682999994, 2021-01-15T19:10:44.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:10:44.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.782 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.783 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:10:44.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.783 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:44.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:44.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:10:44.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:10:44.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:10:44.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:10:44.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:10:44.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:10:44.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:10:44.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:10:44.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:10:44.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:10:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:10:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:10:44.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:10:44.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:10:44.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:10:44.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:10:44.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:10:44.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:10:44.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:10:44.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:10:44.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:10:44.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:10:44.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:10:44.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:10:44.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:10:44.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.798 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:44.798 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:44.799 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 92.105874188000001, 2021-01-15T19:10:44.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:10:44.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:44.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.800 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:10:44.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:44.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:44.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:10:44.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:10:44.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:10:44.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:10:44.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:10:44.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:10:44.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:10:44.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:10:44.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:10:44.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:10:44.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:10:44.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:10:44.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:10:44.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:10:44.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:10:44.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:44.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:10:44.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:10:44.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:10:44.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:10:44.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:10:44.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:10:44.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:10:44.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:10:44.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.814 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.815 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:10:44.815 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:44.815 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 90.123476468000007, 2021-01-15T19:10:44.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:10:44.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:44.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:10:44.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.817 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:44.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:44.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:10:44.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:10:44.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:10:44.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:10:44.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:10:44.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:10:44.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:10:44.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:10:44.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:10:44.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:10:44.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:10:44.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:44.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:10:44.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:10:44.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:10:44.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:10:44.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:44.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:10:44.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:10:44.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:10:44.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:10:44.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:10:44.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:10:44.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:10:44.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:10:44.830 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.830 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.831 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.831 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:10:44.831 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:44.832 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.976723410000002, 2021-01-15T19:10:44.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:10:44.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:44.832 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:10:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:10:44.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:10:44.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:10:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:10:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:10:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:10:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:10:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:10:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:10:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:10:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:10:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:10:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:10:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:10:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:10:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:10:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:10:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:10:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:10:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:10:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:10:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.847 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:10:44.847 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:44.847 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 22.976560478, 2021-01-15T19:10:44.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:10:44.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:44.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:44.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:44.848 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:44.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:10:44.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.849 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:44.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:44.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:10:44.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:10:44.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:10:44.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:10:44.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:10:44.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:10:44.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:10:44.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:10:44.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:10:44.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:10:44.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:10:44.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:44.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:10:44.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:10:44.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:10:44.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:10:44.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:44.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:10:44.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:10:44.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:10:44.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:10:44.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:10:44.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:10:44.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:10:44.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:10:44.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.863 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:10:44.863 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:44.864 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 17.202887840999999, 2021-01-15T19:10:44.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:10:44.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:44.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:44.864 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:44.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:44.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:44.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:44.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:44.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:44.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:10:44.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:44.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:10:44.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:10:44.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:44.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:10:44.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:10:44.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:44.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:10:44.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:10:44.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:44.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:10:44.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:10:44.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:10:44.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:10:44.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:44.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:10:44.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:10:44.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:44.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:10:44.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:10:44.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:44.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:10:44.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:10:44.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:44.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:44.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:44.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:10:44.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:10:44.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:44.876 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:44.877 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:44.877 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:44.877 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:10:44.878 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 2, 3, 4, 5, 7, 6] 2021-01-15T19:10:44.878 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2021-01-15T19:10:44.878 INFO:teuthology.orchestra.run:waiting for 300 2021-01-15T19:10:44.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:44.880 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:44.877+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:44.914 INFO:tasks.ceph.osd.0:Stopped 2021-01-15T19:10:44.914 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2021-01-15T19:10:44.915 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:10:44.915 INFO:tasks.thrashosds.thrasher:remote for exp osd 1 is ubuntu@smithi001.front.sepia.ceph.com 2021-01-15T19:10:44.916 DEBUG:teuthology.orchestra.run.smithi001:> type ceph-objectstore-tool 2021-01-15T19:10:44.968 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2021-01-15T19:10:44.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2021-01-15T19:10:44.978 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:44.977+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:45.016 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:45.016 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:45.017 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:45.017 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:45.026 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:45.111 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:45.110+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:45.180 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:45.179+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:45.192 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:45.193 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:45.193 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:45.193 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:45.194 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:45.202 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:45.262 INFO:tasks.rados.rados.0.smithi001.stdout:758: done (2 left) 2021-01-15T19:10:45.263 INFO:tasks.rados.rados.0.smithi001.stdout:759: read oid 1 snap -1 2021-01-15T19:10:45.263 INFO:tasks.rados.rados.0.smithi001.stdout:759: expect (ObjNum 0 snap 0 seq_num 0) 2021-01-15T19:10:45.263 INFO:tasks.rados.rados.0.smithi001.stdout:760: snap_remove snap 64 2021-01-15T19:10:45.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:10:45.286 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:45.285+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:45.376 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:45.376 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:45.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:10:45.377 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:45.378 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 107.19182772000001, 2021-01-15T19:10:45.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:10:45.378 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.378 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.379 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:10:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:45.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:10:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:10:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:10:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:10:45.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:10:45.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:10:45.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:10:45.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:10:45.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:10:45.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:10:45.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:10:45.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:45.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:10:45.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:10:45.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:10:45.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:10:45.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:10:45.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:10:45.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:10:45.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:10:45.392 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.392 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.393 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.393 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:45.393 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:45.394 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 97.189244954000003, 2021-01-15T19:10:45.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:10:45.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.394 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.394 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.395 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:10:45.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:45.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:45.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:10:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:10:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:10:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:10:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:10:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:10:45.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:10:45.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:10:45.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:10:45.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:10:45.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:10:45.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:10:45.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:10:45.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:10:45.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:10:45.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.406 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.407 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:45.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:45.407 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 97.189082013000004, 2021-01-15T19:10:45.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:10:45.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:10:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:45.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:10:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:10:45.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:10:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:10:45.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:10:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:10:45.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:10:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:10:45.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:10:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:10:45.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:10:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:10:45.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:10:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:10:45.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:10:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:45.420 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.028039367000005, 2021-01-15T19:10:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:10:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.421 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:10:45.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:45.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:45.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:10:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:10:45.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:10:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:10:45.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:10:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:10:45.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:10:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:10:45.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:10:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:10:45.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:10:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:10:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:10:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:10:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:10:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:10:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:10:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:10:45.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:10:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:10:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:10:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:10:45.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:10:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:10:45.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:10:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.027931910999996, 2021-01-15T19:10:45.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:10:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:10:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:10:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:10:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:10:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:10:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:10:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:10:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:10:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:10:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:10:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:10:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:10:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:10:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:10:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:10:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:10:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:10:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:10:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:10:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:10:45.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:10:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:10:45.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:10:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:10:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:10:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:10:45.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:10:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:10:45.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.456 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.894865045000003, 2021-01-15T19:10:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:10:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.457 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:10:45.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:45.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:45.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:10:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:10:45.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:10:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:10:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:10:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:10:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:10:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:10:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:10:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:10:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:10:45.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:10:45.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:10:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:10:45.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:10:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:10:45.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:10:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:10:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:10:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:10:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:10:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:10:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:10:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:10:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:10:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:45.475 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.998897361999994, 2021-01-15T19:10:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:10:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:45.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:45.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:10:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:10:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:10:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:10:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:10:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:10:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:10:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:10:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:10:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:10:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:10:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:10:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:10:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:10:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:10:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:10:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.998767232000006, 2021-01-15T19:10:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:10:45.490 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.491 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:10:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:10:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:10:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:10:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:10:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:10:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:10:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:10:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:10:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:10:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:10:45.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:10:45.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:10:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:10:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:10:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:10:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:10:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:45.504 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.998681822999998, 2021-01-15T19:10:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:10:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:10:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:10:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:10:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:10:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:10:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:10:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:10:45.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:10:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:10:45.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:10:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:10:45.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:10:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:10:45.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:10:45.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:10:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:10:45.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:10:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:10:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.167507435000005, 2021-01-15T19:10:45.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:10:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:10:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:10:45.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:10:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:10:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:10:45.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:10:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:10:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:10:45.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:10:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:10:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:10:45.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:10:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:10:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:10:45.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:10:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:10:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:10:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:10:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:10:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:10:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:10:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:10:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:10:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:10:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:10:45.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:10:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:10:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:10:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:10:45.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:10:45.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.546 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:10:45.546 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:45.547 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.167418785999999, 2021-01-15T19:10:45.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:10:45.547 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.547 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.547 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.548 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.548 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:10:45.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.548 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:45.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:45.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:10:45.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:10:45.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:10:45.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:10:45.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:45.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:10:45.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:10:45.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:10:45.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:10:45.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:10:45.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:10:45.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:45.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:10:45.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:10:45.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:10:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:10:45.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:45.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:10:45.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:10:45.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:10:45.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:10:45.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:10:45.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:10:45.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:10:45.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:10:45.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:10:45.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:10:45.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:10:45.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:10:45.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:10:45.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:10:45.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.565 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.566 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:10:45.566 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:45.566 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.183910728000001, 2021-01-15T19:10:45.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:10:45.567 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.567 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.567 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.568 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:10:45.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.568 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:45.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:45.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:10:45.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:10:45.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:10:45.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:10:45.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:10:45.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:10:45.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:10:45.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:10:45.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:10:45.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:10:45.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:10:45.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:45.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:10:45.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:10:45.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:10:45.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:10:45.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:10:45.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:10:45.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:10:45.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:10:45.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:10:45.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:10:45.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:10:45.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:10:45.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:10:45.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:10:45.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.584 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:10:45.585 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:45.585 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.183778653000005, 2021-01-15T19:10:45.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:10:45.585 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.585 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.586 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.586 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:10:45.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.587 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:45.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:45.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:10:45.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:10:45.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:10:45.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:10:45.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:10:45.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:10:45.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:10:45.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:10:45.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:10:45.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:10:45.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:10:45.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:45.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:10:45.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:10:45.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:10:45.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:10:45.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:45.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:10:45.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:10:45.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:10:45.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:10:45.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:45.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:10:45.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:10:45.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:10:45.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:10:45.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:10:45.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:10:45.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:10:45.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:10:45.602 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.603 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:10:45.604 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:45.604 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.180643136, 2021-01-15T19:10:45.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:10:45.604 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.605 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:10:45.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:45.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:45.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:10:45.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:10:45.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:10:45.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:10:45.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:10:45.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:10:45.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:10:45.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:10:45.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:10:45.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:10:45.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:10:45.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:45.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:10:45.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:10:45.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:10:45.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:10:45.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:10:45.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:10:45.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:10:45.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:10:45.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:10:45.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:10:45.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:10:45.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:10:45.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:10:45.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:10:45.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.624 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:10:45.625 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:45.625 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.180539109999998, 2021-01-15T19:10:45.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:10:45.626 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.627 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:10:45.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:45.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:45.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:10:45.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:10:45.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:10:45.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:10:45.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:10:45.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:10:45.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:10:45.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:10:45.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:10:45.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:10:45.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:10:45.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:45.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:10:45.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:10:45.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:10:45.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:10:45.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:45.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:10:45.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:10:45.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:10:45.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:10:45.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:45.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:10:45.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:10:45.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:10:45.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:10:45.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:10:45.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:10:45.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:10:45.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:10:45.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.643 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.643 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.644 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:45.644 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:45.644 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.169552361000001, 2021-01-15T19:10:45.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:10:45.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.646 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:10:45.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:45.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:45.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:10:45.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:10:45.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:10:45.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:10:45.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:10:45.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:10:45.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:10:45.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:45.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:10:45.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:10:45.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:10:45.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:45.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:10:45.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:10:45.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:10:45.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:10:45.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:10:45.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:10:45.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:10:45.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:10:45.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.659 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:45.659 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:45.659 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.169456840000002, 2021-01-15T19:10:45.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:10:45.659 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.660 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.660 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:10:45.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.661 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:45.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:45.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:10:45.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:10:45.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:10:45.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:10:45.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:10:45.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:10:45.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:10:45.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:10:45.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:10:45.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:10:45.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:10:45.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:45.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:10:45.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:10:45.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:10:45.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:10:45.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:10:45.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:10:45.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:10:45.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:10:45.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.674 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:45.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:45.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.169345694, 2021-01-15T19:10:45.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:10:45.675 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.675 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:45.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.676 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.676 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:10:45.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.676 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:45.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:45.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:10:45.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:10:45.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:10:45.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:10:45.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:10:45.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:10:45.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:10:45.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:45.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:10:45.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:10:45.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:10:45.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:45.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:10:45.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:10:45.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:10:45.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:10:45.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:10:45.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:10:45.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:10:45.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:10:45.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.688 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:10:45.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:45.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.077571372000001, 2021-01-15T19:10:45.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:10:45.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.691 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.691 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:10:45.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.692 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:45.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:45.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:10:45.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:10:45.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:10:45.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:10:45.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:10:45.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:10:45.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:10:45.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:10:45.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:10:45.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:10:45.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:10:45.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:10:45.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:10:45.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:10:45.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:10:45.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:10:45.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:10:45.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:10:45.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:10:45.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:10:45.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:10:45.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.705 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:10:45.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:45.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.076288233, 2021-01-15T19:10:45.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:10:45.708 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:45.709 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:45.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:45.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:45.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:45.710 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:10:45.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.711 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:45.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:45.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:45.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:45.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:45.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:45.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:10:45.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:45.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:10:45.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:10:45.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:45.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:10:45.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:10:45.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:45.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:10:45.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:10:45.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:45.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:10:45.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:10:45.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:45.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:10:45.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:10:45.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:45.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:10:45.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:10:45.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:45.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:10:45.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:10:45.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:45.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:10:45.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:10:45.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:45.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:10:45.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:10:45.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:45.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:45.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:45.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:10:45.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:10:45.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.724 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:45.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:45.725 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:45.726 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:45.727 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:45.725+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:45.827 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:45.826+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:45.896 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:45.896 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:45.897 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:45.897 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:45.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:45.946 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:45.945+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:45.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:46.029 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:46.027+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:46.071 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:46.071 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:46.072 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:46.072 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:46.073 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:46.082 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:46.128 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:46.130 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:46.129+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:46.146 INFO:teuthology.orchestra.run.smithi001.stdout:3.fs1 2021-01-15T19:10:46.147 INFO:teuthology.orchestra.run.smithi001.stdout:3.as1 2021-01-15T19:10:46.147 INFO:teuthology.orchestra.run.smithi001.stdout:3.17s2 2021-01-15T19:10:46.147 INFO:teuthology.orchestra.run.smithi001.stdout:3.15s1 2021-01-15T19:10:46.147 INFO:teuthology.orchestra.run.smithi001.stdout:3.14s2 2021-01-15T19:10:46.148 INFO:teuthology.orchestra.run.smithi001.stdout:3.12s0 2021-01-15T19:10:46.148 INFO:teuthology.orchestra.run.smithi001.stdout:3.19s2 2021-01-15T19:10:46.148 INFO:teuthology.orchestra.run.smithi001.stdout:3.11s2 2021-01-15T19:10:46.148 INFO:teuthology.orchestra.run.smithi001.stdout:3.10s0 2021-01-15T19:10:46.148 INFO:teuthology.orchestra.run.smithi001.stdout:2.f 2021-01-15T19:10:46.149 INFO:teuthology.orchestra.run.smithi001.stdout:2.d 2021-01-15T19:10:46.149 INFO:teuthology.orchestra.run.smithi001.stdout:2.7 2021-01-15T19:10:46.149 INFO:teuthology.orchestra.run.smithi001.stdout:2.2 2021-01-15T19:10:46.149 INFO:teuthology.orchestra.run.smithi001.stdout:1.e 2021-01-15T19:10:46.149 INFO:teuthology.orchestra.run.smithi001.stdout:1.d 2021-01-15T19:10:46.150 INFO:teuthology.orchestra.run.smithi001.stdout:1.c 2021-01-15T19:10:46.150 INFO:teuthology.orchestra.run.smithi001.stdout:1.b 2021-01-15T19:10:46.150 INFO:teuthology.orchestra.run.smithi001.stdout:3.7s2 2021-01-15T19:10:46.150 INFO:teuthology.orchestra.run.smithi001.stdout:1.a 2021-01-15T19:10:46.150 INFO:teuthology.orchestra.run.smithi001.stdout:1.f 2021-01-15T19:10:46.151 INFO:teuthology.orchestra.run.smithi001.stdout:1.9 2021-01-15T19:10:46.151 INFO:teuthology.orchestra.run.smithi001.stdout:1.8 2021-01-15T19:10:46.151 INFO:teuthology.orchestra.run.smithi001.stdout:3.6s2 2021-01-15T19:10:46.151 INFO:teuthology.orchestra.run.smithi001.stdout:1.7 2021-01-15T19:10:46.151 INFO:teuthology.orchestra.run.smithi001.stdout:1.3 2021-01-15T19:10:46.152 INFO:teuthology.orchestra.run.smithi001.stdout:3.4s1 2021-01-15T19:10:46.152 INFO:teuthology.orchestra.run.smithi001.stdout:1.1 2021-01-15T19:10:46.152 INFO:teuthology.orchestra.run.smithi001.stdout:1.13 2021-01-15T19:10:46.152 INFO:teuthology.orchestra.run.smithi001.stdout:1.11 2021-01-15T19:10:46.152 INFO:teuthology.orchestra.run.smithi001.stdout:1.10 2021-01-15T19:10:46.153 INFO:teuthology.orchestra.run.smithi001.stdout:3.ds2 2021-01-15T19:10:46.153 INFO:teuthology.orchestra.run.smithi001.stdout:1.0 2021-01-15T19:10:46.153 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:46.170 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:46.191 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:46.209 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:46.209 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:46.213 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:46.219 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:46.231 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:46.230+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:46.233 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:46.233 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:10:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:10:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:46.234 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:46.236 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:46.236 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:46.269 INFO:tasks.rados.rados.0.smithi001.stdout:760: done (3 left) 2021-01-15T19:10:46.269 INFO:tasks.rados.rados.0.smithi001.stdout:761: delete oid 24 current snap is 80 2021-01-15T19:10:46.269 INFO:tasks.rados.rados.0.smithi001.stdout:754: finishing write tid 5 to smithi00140323-7 2021-01-15T19:10:46.270 INFO:tasks.rados.rados.0.smithi001.stdout:756: finishing write tid 1 to smithi00140323-34 2021-01-15T19:10:46.270 INFO:tasks.rados.rados.0.smithi001.stdout:756: finishing write tid 2 to smithi00140323-34 2021-01-15T19:10:46.270 INFO:tasks.rados.rados.0.smithi001.stdout:754: finishing write tid 6 to smithi00140323-7 2021-01-15T19:10:46.270 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 378 (ObjNum 265 snap 80 seq_num 265) dirty exists 2021-01-15T19:10:46.271 INFO:tasks.rados.rados.0.smithi001.stdout:754: left oid 7 (ObjNum 265 snap 80 seq_num 265) 2021-01-15T19:10:46.271 INFO:tasks.rados.rados.0.smithi001.stdout:756: finishing write tid 3 to smithi00140323-34 2021-01-15T19:10:46.271 INFO:tasks.rados.rados.0.smithi001.stdout:756: finishing write tid 4 to smithi00140323-34 2021-01-15T19:10:46.271 INFO:tasks.rados.rados.0.smithi001.stdout:756: finishing write tid 5 to smithi00140323-34 2021-01-15T19:10:46.271 INFO:tasks.rados.rados.0.smithi001.stdout:756: finishing write tid 6 to smithi00140323-34 2021-01-15T19:10:46.272 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 34 v 383 (ObjNum 266 snap 80 seq_num 266) dirty exists 2021-01-15T19:10:46.272 INFO:tasks.rados.rados.0.smithi001.stdout:756: left oid 34 (ObjNum 266 snap 80 seq_num 266) 2021-01-15T19:10:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:10:46.279 INFO:tasks.rados.rados.0.smithi001.stdout:754: done (3 left) 2021-01-15T19:10:46.280 INFO:tasks.rados.rados.0.smithi001.stdout:756: done (2 left) 2021-01-15T19:10:46.280 INFO:tasks.rados.rados.0.smithi001.stdout:761: done (1 left) 2021-01-15T19:10:46.280 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 3 current snap is 80 2021-01-15T19:10:46.280 INFO:tasks.rados.rados.0.smithi001.stdout:762: seq_num 268 ranges {2277376=491520,2768896=745472,3514368=745472,4259840=114688} 2021-01-15T19:10:46.295 INFO:tasks.rados.rados.0.smithi001.stdout:762: writing smithi00140323-3 from 2277376 to 2768896 tid 1 2021-01-15T19:10:46.299 INFO:tasks.rados.rados.0.smithi001.stdout:762: writing smithi00140323-3 from 2768896 to 3514368 tid 2 2021-01-15T19:10:46.303 INFO:tasks.rados.rados.0.smithi001.stdout:762: writing smithi00140323-3 from 3514368 to 4259840 tid 3 2021-01-15T19:10:46.304 INFO:tasks.rados.rados.0.smithi001.stdout:762: writing smithi00140323-3 from 4259840 to 4374528 tid 4 2021-01-15T19:10:46.304 INFO:tasks.rados.rados.0.smithi001.stdout:762: finishing write tid 1 to smithi00140323-3 2021-01-15T19:10:46.305 INFO:tasks.rados.rados.0.smithi001.stdout:763: copy_from oid 29 from oid 36 current snap is 80 2021-01-15T19:10:46.305 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 31 current snap is 80 2021-01-15T19:10:46.305 INFO:tasks.rados.rados.0.smithi001.stdout:764: seq_num 269 ranges {1196032=507904,1703936=630784,2334720=557056,2891776=417792} 2021-01-15T19:10:46.313 INFO:tasks.rados.rados.0.smithi001.stdout:764: writing smithi00140323-31 from 1196032 to 1703936 tid 1 2021-01-15T19:10:46.317 INFO:tasks.rados.rados.0.smithi001.stdout:764: writing smithi00140323-31 from 1703936 to 2334720 tid 2 2021-01-15T19:10:46.320 INFO:tasks.rados.rados.0.smithi001.stdout:764: writing smithi00140323-31 from 2334720 to 2891776 tid 3 2021-01-15T19:10:46.323 INFO:tasks.rados.rados.0.smithi001.stdout:764: writing smithi00140323-31 from 2891776 to 3309568 tid 4 2021-01-15T19:10:46.323 INFO:tasks.rados.rados.0.smithi001.stdout:765: snap_create 2021-01-15T19:10:46.323 INFO:tasks.rados.rados.0.smithi001.stdout:762: finishing write tid 2 to smithi00140323-3 2021-01-15T19:10:46.323 INFO:tasks.rados.rados.0.smithi001.stdout:762: finishing write tid 3 to smithi00140323-3 2021-01-15T19:10:46.324 INFO:tasks.rados.rados.0.smithi001.stdout:762: finishing write tid 4 to smithi00140323-3 2021-01-15T19:10:46.324 INFO:tasks.rados.rados.0.smithi001.stdout:762: finishing write tid 5 to smithi00140323-3 2021-01-15T19:10:46.324 INFO:tasks.rados.rados.0.smithi001.stdout:762: finishing write tid 6 to smithi00140323-3 2021-01-15T19:10:46.324 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 3 v 116 (ObjNum 268 snap 80 seq_num 268) dirty exists 2021-01-15T19:10:46.325 INFO:tasks.rados.rados.0.smithi001.stdout:762: left oid 3 (ObjNum 268 snap 80 seq_num 268) 2021-01-15T19:10:46.325 INFO:tasks.rados.rados.0.smithi001.stdout:764: finishing write tid 1 to smithi00140323-31 2021-01-15T19:10:46.325 INFO:tasks.rados.rados.0.smithi001.stdout:764: finishing write tid 2 to smithi00140323-31 2021-01-15T19:10:46.326 INFO:tasks.rados.rados.0.smithi001.stdout:764: finishing write tid 3 to smithi00140323-31 2021-01-15T19:10:46.326 INFO:tasks.rados.rados.0.smithi001.stdout:764: finishing write tid 4 to smithi00140323-31 2021-01-15T19:10:46.327 INFO:tasks.rados.rados.0.smithi001.stdout:764: finishing write tid 5 to smithi00140323-31 2021-01-15T19:10:46.328 INFO:tasks.rados.rados.0.smithi001.stdout:764: finishing write tid 6 to smithi00140323-31 2021-01-15T19:10:46.329 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 31 v 282 (ObjNum 269 snap 80 seq_num 269) dirty exists 2021-01-15T19:10:46.329 INFO:tasks.rados.rados.0.smithi001.stdout:764: left oid 31 (ObjNum 269 snap 80 seq_num 269) 2021-01-15T19:10:46.332 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:46.331+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:46.367 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:46.367 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:46.376 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:46.460 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:46.459+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:46.539 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:46.537+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:46.868 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:46.869 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:46.870 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:46.866+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:46.871 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 3.19s2 --file /var/log/ceph/exp.3.19s2.1 2021-01-15T19:10:46.872 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:46.968 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:46.967+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:47.034 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:47.034 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:47.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:47.085 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:47.084+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:47.170 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:47.169+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:47.210 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:47.210 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:47.219 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:47.267 INFO:tasks.rados.rados.0.smithi001.stdout:762: done (4 left) 2021-01-15T19:10:47.268 INFO:tasks.rados.rados.0.smithi001.stdout:764: done (3 left) 2021-01-15T19:10:47.268 INFO:tasks.rados.rados.0.smithi001.stdout:765: done (2 left) 2021-01-15T19:10:47.268 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 16 current snap is 81 2021-01-15T19:10:47.268 INFO:tasks.rados.rados.0.smithi001.stdout:766: seq_num 270 ranges {1490944=524288,2015232=516096,2531328=761856,3293184=327680} 2021-01-15T19:10:47.279 INFO:tasks.rados.rados.0.smithi001.stdout:766: writing smithi00140323-16 from 1490944 to 2015232 tid 1 2021-01-15T19:10:47.282 INFO:tasks.rados.rados.0.smithi001.stdout:766: writing smithi00140323-16 from 2015232 to 2531328 tid 2 2021-01-15T19:10:47.288 INFO:tasks.rados.rados.0.smithi001.stdout:766: writing smithi00140323-16 from 2531328 to 3293184 tid 3 2021-01-15T19:10:47.290 INFO:tasks.rados.rados.0.smithi001.stdout:766: writing smithi00140323-16 from 3293184 to 3620864 tid 4 2021-01-15T19:10:47.291 INFO:tasks.rados.rados.0.smithi001.stdout:767: snap_create 2021-01-15T19:10:47.291 INFO:tasks.rados.rados.0.smithi001.stdout:766: finishing write tid 1 to smithi00140323-16 2021-01-15T19:10:47.291 INFO:tasks.rados.rados.0.smithi001.stdout:766: finishing write tid 2 to smithi00140323-16 2021-01-15T19:10:47.296 INFO:tasks.rados.rados.0.smithi001.stdout:766: finishing write tid 3 to smithi00140323-16 2021-01-15T19:10:47.296 INFO:tasks.rados.rados.0.smithi001.stdout:766: finishing write tid 4 to smithi00140323-16 2021-01-15T19:10:47.297 INFO:tasks.rados.rados.0.smithi001.stdout:766: finishing write tid 5 to smithi00140323-16 2021-01-15T19:10:47.298 INFO:tasks.rados.rados.0.smithi001.stdout:766: finishing write tid 6 to smithi00140323-16 2021-01-15T19:10:47.298 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 191 (ObjNum 270 snap 81 seq_num 270) dirty exists 2021-01-15T19:10:47.299 INFO:tasks.rados.rados.0.smithi001.stdout:766: left oid 16 (ObjNum 270 snap 81 seq_num 270) 2021-01-15T19:10:47.303 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:47.302+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:47.371 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:47.370+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:47.386 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:47.386 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:47.396 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:47.473 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:47.471+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:47.560 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:47.561 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:47.570 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:47.574 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:47.573+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:47.676 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:47.674+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:47.704 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:47.705 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:47.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:47.776 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:47.776+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:47.877 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:47.877+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:47.886 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:47.886 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:47.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:47.979 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:47.978+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:48.027 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:48.028 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:48.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:48.051 INFO:teuthology.orchestra.run.smithi001.stderr:Exporting 3.19s2 info 3.19s2( v 155'64 (0'0,155'64] local-lis/les=57/58 n=6 ec=52/21 lis/c=57/57 les/c/f=58/58/0 sis=57) 2021-01-15T19:10:48.053 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:9c47a2ed:::smithi00140323-38:1b# 2021-01-15T19:10:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:9c47a2ed:::smithi00140323-38:46# 2021-01-15T19:10:48.060 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:9c47a2ed:::smithi00140323-38:4b# 2021-01-15T19:10:48.064 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:9c47a2ed:::smithi00140323-38:72# 2021-01-15T19:10:48.066 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:9c47a2ed:::smithi00140323-38:7d# 2021-01-15T19:10:48.070 INFO:teuthology.orchestra.run.smithi001.stderr:Read 2#3:9c47a2ed:::smithi00140323-38:head# 2021-01-15T19:10:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:Export successful 2021-01-15T19:10:48.079 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:48.079+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:48.172 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:48.172 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:48.180 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:48.180+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:48.181 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:48.204 INFO:tasks.rados.rados.0.smithi001.stdout:766: done (3 left) 2021-01-15T19:10:48.206 INFO:tasks.rados.rados.0.smithi001.stdout:767: done (2 left) 2021-01-15T19:10:48.206 INFO:tasks.rados.rados.0.smithi001.stdout:768: rmattr oid 11 current snap is 82 2021-01-15T19:10:48.207 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 7 current snap is 82 2021-01-15T19:10:48.208 INFO:tasks.rados.rados.0.smithi001.stdout:769: seq_num 271 ranges {8192000=540672,8732672=802816,9535488=516096,10051584=286720} 2021-01-15T19:10:48.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:48.252 INFO:tasks.rados.rados.0.smithi001.stdout:769: writing smithi00140323-7 from 8192000 to 8732672 tid 1 2021-01-15T19:10:48.257 INFO:tasks.rados.rados.0.smithi001.stdout:769: writing smithi00140323-7 from 8732672 to 9535488 tid 2 2021-01-15T19:10:48.260 INFO:tasks.rados.rados.0.smithi001.stdout:769: writing smithi00140323-7 from 9535488 to 10051584 tid 3 2021-01-15T19:10:48.263 INFO:tasks.rados.rados.0.smithi001.stdout:769: writing smithi00140323-7 from 10051584 to 10338304 tid 4 2021-01-15T19:10:48.263 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 385 (ObjNum 1 snap 32581 seq_num 179443904) dirty exists 2021-01-15T19:10:48.263 INFO:tasks.rados.rados.0.smithi001.stdout:768: done (3 left) 2021-01-15T19:10:48.263 INFO:tasks.rados.rados.0.smithi001.stdout:770: snap_create 2021-01-15T19:10:48.273 INFO:tasks.rados.rados.0.smithi001.stdout:769: finishing write tid 1 to smithi00140323-7 2021-01-15T19:10:48.273 INFO:tasks.rados.rados.0.smithi001.stdout:769: finishing write tid 2 to smithi00140323-7 2021-01-15T19:10:48.274 INFO:tasks.rados.rados.0.smithi001.stdout:769: finishing write tid 3 to smithi00140323-7 2021-01-15T19:10:48.275 INFO:tasks.rados.rados.0.smithi001.stdout:769: finishing write tid 4 to smithi00140323-7 2021-01-15T19:10:48.275 INFO:tasks.rados.rados.0.smithi001.stdout:769: finishing write tid 5 to smithi00140323-7 2021-01-15T19:10:48.276 INFO:tasks.rados.rados.0.smithi001.stdout:769: finishing write tid 6 to smithi00140323-7 2021-01-15T19:10:48.276 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 7 v 391 (ObjNum 271 snap 82 seq_num 271) dirty exists 2021-01-15T19:10:48.276 INFO:tasks.rados.rados.0.smithi001.stdout:769: left oid 7 (ObjNum 271 snap 82 seq_num 271) 2021-01-15T19:10:48.281 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:48.280+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:48.285 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:10:48.348 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:48.349 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:48.359 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:48.403 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:48.402+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:48.410 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:48.435 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:48.458 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:48.478 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:48.483 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:48.482+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:48.499 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:48.503 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 129.25173293500001, 2021-01-15T19:10:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:10:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:10:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:10:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:10:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:10:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:10:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:10:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:10:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:10:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:10:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:10:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:10:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:10:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:10:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:10:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:10:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:10:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:10:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:10:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:10:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:10:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:10:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:10:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 129.25154199100001, 2021-01-15T19:10:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:10:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:10:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:10:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:10:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:10:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:10:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:10:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:10:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:10:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:10:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:10:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:10:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:10:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:10:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:10:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:10:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:10:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:10:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:10:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:10:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:10:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:10:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:10:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:10:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:10:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.312589894, 2021-01-15T19:10:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:10:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:10:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:10:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:10:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:10:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:10:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:10:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:10:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:10:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:10:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:10:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:10:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:10:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:10:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:10:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:10:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:10:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:10:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:10:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:10:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:10:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:10:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:10:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.309869433, 2021-01-15T19:10:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:10:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:10:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:10:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:10:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:10:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:10:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:10:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:10:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:10:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:10:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:10:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:10:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:10:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:10:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:10:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:10:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:10:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:10:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:10:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:10:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:10:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:10:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:10:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:10:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:10:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.309756854, 2021-01-15T19:10:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:10:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:10:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:10:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:10:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:10:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:10:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:10:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:10:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:10:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:10:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:10:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:10:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:10:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:10:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:10:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:10:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:10:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:10:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:10:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:10:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:10:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:10:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:10:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.30962694999999, 2021-01-15T19:10:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:10:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:10:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:10:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:10:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:10:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:10:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:10:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:10:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:10:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:10:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:10:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:10:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:10:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:10:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:10:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:10:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:10:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:10:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:10:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:10:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:10:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:10:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:10:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:10:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:10:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.28642168099999, 2021-01-15T19:10:48.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:10:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.602 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:10:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:10:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:10:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:10:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:10:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:10:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:10:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:10:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:10:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:10:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:10:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:10:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:10:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:10:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:10:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:10:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:10:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:10:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:10:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:10:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.286303483, 2021-01-15T19:10:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:10:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:10:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:10:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:10:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:10:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:10:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:10:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:10:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:10:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:10:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:10:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:10:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:10:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:10:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:10:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:10:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:10:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:10:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:10:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:10:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:10:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:10:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:10:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.286190793, 2021-01-15T19:10:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:10:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:10:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:10:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:10:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:10:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:10:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:10:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:10:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:10:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:10:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:10:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:10:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:10:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:10:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:10:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:10:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:10:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:10:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:10:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:10:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:10:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:10:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:10:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:10:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.28058904300001, 2021-01-15T19:10:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:10:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:10:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:10:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:10:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:10:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:10:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:10:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:10:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:10:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:10:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:10:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:10:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:10:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:10:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.272120319, 2021-01-15T19:10:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:10:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:10:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:10:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:10:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:10:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:10:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:10:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:10:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:10:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:10:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:10:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:10:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:10:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:10:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:10:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:10:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:10:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:10:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:10:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:10:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:10:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:10:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:10:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.267601736, 2021-01-15T19:10:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:10:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:10:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:10:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:10:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:10:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:10:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:10:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:10:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:10:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:10:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:10:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:10:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:10:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:10:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:10:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:10:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:10:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:10:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:10:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:10:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:10:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:10:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:10:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.262583967, 2021-01-15T19:10:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:10:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:10:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:10:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:10:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:10:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:10:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:10:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:10:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:10:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:10:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:10:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:10:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:10:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:10:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:10:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:10:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:10:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:10:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:10:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:10:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:10:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:10:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 110.295459889, 2021-01-15T19:10:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:10:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:10:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:10:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:10:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:10:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:10:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:10:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:10:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:10:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:10:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:10:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:10:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:10:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:10:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:10:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:10:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:10:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:10:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:10:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:10:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:10:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:10:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:10:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:10:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:10:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:10:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:10:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:10:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 100.318273241, 2021-01-15T19:10:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:10:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:10:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:10:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:10:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:10:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:10:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:10:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:10:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:10:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:10:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:10:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:10:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:10:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:10:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:10:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:10:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:10:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:10:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:10:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:10:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:10:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:10:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:10:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:10:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:10:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:10:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:10:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 50.287726220000003, 2021-01-15T19:10:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:10:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:10:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:10:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:10:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:10:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:10:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:10:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:10:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:10:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:10:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:10:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:10:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:10:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:10:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:10:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:10:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:10:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:10:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:10:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:10:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:10:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:10:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:10:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:10:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:10:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.297068244999998, 2021-01-15T19:10:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:10:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:10:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:10:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:10:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:10:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:10:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:10:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:10:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:10:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:10:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:10:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:10:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:10:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:10:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:10:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:10:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:10:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:10:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:10:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:10:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:10:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:10:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:10:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:10:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:10:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:10:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.296928712000003, 2021-01-15T19:10:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:10:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:10:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:10:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:10:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:10:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:10:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:10:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:10:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:10:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:10:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:10:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:10:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:10:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:10:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:10:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:10:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:10:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:10:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:10:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:10:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:10:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:10:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:10:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:10:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:10:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:10:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:10:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 45.296813399000001, 2021-01-15T19:10:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:10:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:10:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:10:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:10:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:10:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:10:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:10:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:10:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:10:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:10:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:10:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:10:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:10:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:10:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:10:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:10:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:10:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:10:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:10:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:10:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:10:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:10:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:10:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:10:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:10:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:10:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:10:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 9.1833421029999993, 2021-01-15T19:10:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:10:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:10:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:10:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:10:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:10:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:10:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:10:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:10:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:10:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:10:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:10:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:10:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:10:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:10:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:10:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:10:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:10:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:48.817 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:48.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:10:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:10:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:48.820 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:48.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:48.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:48.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:48.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:48.821 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:48.822 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:48.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:48.822 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:48.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 3.19s2 2021-01-15T19:10:48.823 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:48.818+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:48.824 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:48.920 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:48.918+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:48.960 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:48.961 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:48.970 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:49.021 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:49.019+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:49.094 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:49.095 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:49.095 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:49.095 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:49.096 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:49.104 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:49.122 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:49.120+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:49.215 INFO:tasks.rados.rados.0.smithi001.stdout:769: done (3 left) 2021-01-15T19:10:49.215 INFO:tasks.rados.rados.0.smithi001.stdout:770: done (2 left) 2021-01-15T19:10:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:771: read oid 16 snap -1 2021-01-15T19:10:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:771: expect (ObjNum 270 snap 81 seq_num 270) 2021-01-15T19:10:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:772: copy_from oid 45 from oid 30 current snap is 83 2021-01-15T19:10:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 22 current snap is 83 2021-01-15T19:10:49.216 INFO:tasks.rados.rados.0.smithi001.stdout:773: seq_num 272 ranges {2564096=565248,3129344=688128,3817472=729088,4546560=188416} 2021-01-15T19:10:49.217 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:10:49.222 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:49.221+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:49.235 INFO:tasks.rados.rados.0.smithi001.stdout:773: writing smithi00140323-22 from 2564096 to 3129344 tid 1 2021-01-15T19:10:49.239 INFO:tasks.rados.rados.0.smithi001.stdout:773: writing smithi00140323-22 from 3129344 to 3817472 tid 2 2021-01-15T19:10:49.241 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:49.241 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:49.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:49.242 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:49.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.242 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:695 3.7s0 3:e0e4bbc9:::smithi00140323-39:head [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e70)", 2021-01-15T19:10:49.242 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:49.242 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.056654557, 2021-01-15T19:10:49.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001, 2021-01-15T19:10:49.243 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.243 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.244 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.244 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.244 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 695 2021-01-15T19:10:49.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181278+0000", 2021-01-15T19:10:49.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8099999999999999e-05 2021-01-15T19:10:49.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181306+0000", 2021-01-15T19:10:49.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6339999999999999e-06 2021-01-15T19:10:49.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181310+0000", 2021-01-15T19:10:49.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9399999999999999e-06 2021-01-15T19:10:49.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181316+0000", 2021-01-15T19:10:49.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2021-01-15T19:10:49.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181318+0000", 2021-01-15T19:10:49.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3636e-05 2021-01-15T19:10:49.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181362+0000", 2021-01-15T19:10:49.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.181e-06 2021-01-15T19:10:49.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:49.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.181367+0000", 2021-01-15T19:10:49.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016185887999999999 2021-01-15T19:10:49.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197553+0000", 2021-01-15T19:10:49.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3654999999999998e-05 2021-01-15T19:10:49.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.197637+0000", 2021-01-15T19:10:49.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079984500000000007 2021-01-15T19:10:49.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:58.205635+0000", 2021-01-15T19:10:49.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024356907000000001 2021-01-15T19:10:49.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.256 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.256 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:49.257 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:49.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.054071791, 2021-01-15T19:10:49.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:10:49.257 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:10:49.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:49.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:10:49.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:10:49.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:10:49.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:10:49.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:10:49.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:10:49.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:10:49.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:10:49.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:10:49.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:10:49.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:10:49.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:10:49.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:10:49.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:10:49.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:10:49.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:10:49.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:10:49.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:49.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.05390885, 2021-01-15T19:10:49.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:10:49.270 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.270 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.271 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:10:49.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.271 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:49.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:10:49.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:10:49.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:10:49.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:10:49.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:10:49.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:10:49.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:10:49.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:10:49.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:10:49.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:10:49.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:10:49.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:10:49.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:10:49.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:10:49.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:10:49.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:10:49.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.281 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.281 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.282 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:49.282 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:49.282 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.892866204000001, 2021-01-15T19:10:49.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:10:49.283 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.283 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.283 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.284 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:10:49.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.284 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:49.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:10:49.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:10:49.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:10:49.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:10:49.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:10:49.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:10:49.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:10:49.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:10:49.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:10:49.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:10:49.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:10:49.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:10:49.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:10:49.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:10:49.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:10:49.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:10:49.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:10:49.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:10:49.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:10:49.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:10:49.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:10:49.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:10:49.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:10:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:10:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:10:49.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:10:49.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:49.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:49.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.892758748000006, 2021-01-15T19:10:49.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:10:49.300 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.301 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:10:49.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.302 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:49.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:10:49.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:10:49.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:10:49.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:10:49.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:10:49.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:10:49.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:10:49.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:10:49.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:10:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:10:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:10:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:10:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:10:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:10:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:10:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:10:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:10:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:10:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:10:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:10:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:10:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:10:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:10:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:10:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:10:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:10:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:10:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:10:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:49.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.759691881999998, 2021-01-15T19:10:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:10:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:10:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:10:49.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:10:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:10:49.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:10:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:10:49.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:10:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:10:49.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:10:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:10:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:10:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:10:49.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:10:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:10:49.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:10:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:10:49.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:10:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:10:49.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:10:49.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:10:49.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:10:49.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:10:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:10:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:10:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:10:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:10:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:10:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.863724199000004, 2021-01-15T19:10:49.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:10:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.337 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:10:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:10:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:10:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:10:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:10:49.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:10:49.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:10:49.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:10:49.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:10:49.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:10:49.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:49.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:10:49.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:10:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:10:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:10:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:10:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:10:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:10:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:49.349 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.863594069000001, 2021-01-15T19:10:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:10:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.350 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:10:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:10:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:10:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:10:49.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:10:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:10:49.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:10:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:10:49.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:10:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:10:49.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:10:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:10:49.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:10:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:10:49.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:10:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:10:49.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:10:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:10:49.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:10:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.863508659999994, 2021-01-15T19:10:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:10:49.363 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.364 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:10:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:10:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:10:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:10:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:10:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:10:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:10:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:10:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:10:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:10:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:10:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:10:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:10:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:10:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:10:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:10:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:10:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:10:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:10:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:49.376 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.032334272, 2021-01-15T19:10:49.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:10:49.377 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.377 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.377 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.377 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:10:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:10:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:10:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:10:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:10:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:10:49.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:10:49.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:10:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:10:49.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:10:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:10:49.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:10:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:10:49.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:10:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:10:49.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:10:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:10:49.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:10:49.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:10:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:10:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:10:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:10:49.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:10:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:10:49.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:10:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:10:49.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:10:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:10:49.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:10:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:10:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:10:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:10:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.032245622999994, 2021-01-15T19:10:49.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:10:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.397 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:10:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:10:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:10:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:10:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:10:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:10:49.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:10:49.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:10:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:10:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:10:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:10:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:10:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:10:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:49.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:10:49.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:10:49.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:10:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:10:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:10:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:10:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:10:49.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:10:49.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:10:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:10:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:10:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:10:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:10:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:10:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:10:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:10:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:10:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:10:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:10:49.415 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.048737564999996, 2021-01-15T19:10:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:10:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:10:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:10:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:10:49.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:10:49.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:10:49.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:10:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:10:49.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:10:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:10:49.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:10:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:10:49.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:10:49.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:10:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:10:49.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:10:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:10:49.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:10:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:10:49.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:10:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:10:49.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:10:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:10:49.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:10:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:10:49.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:10:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:10:49.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:10:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:10:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.04860549, 2021-01-15T19:10:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:10:49.433 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.434 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:10:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:10:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:10:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:10:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:10:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:10:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:10:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:10:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:10:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:10:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:10:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:10:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:10:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:10:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:10:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:10:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:10:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:10:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:10:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:10:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:10:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:10:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:10:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:10:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:10:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:10:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:10:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:10:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:10:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:10:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:49.451 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.045469972999996, 2021-01-15T19:10:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:10:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.452 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:10:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:10:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:10:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:10:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:10:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:10:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:10:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:10:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:10:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:10:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:10:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:10:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:10:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:49.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:10:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:10:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:10:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:10:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:10:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:10:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:10:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:10:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:10:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:10:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:10:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:10:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:10:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:10:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:10:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 71.045365946999993, 2021-01-15T19:10:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:10:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.469 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:10:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:10:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:10:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:10:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:10:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:10:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:10:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:10:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:10:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:10:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:10:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:10:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:10:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:49.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:10:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:10:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:10:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:10:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:10:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:10:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:10:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:10:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:10:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:10:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:10:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:10:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:10:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:10:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:10:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:10:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.485 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:49.486 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.034379198000003, 2021-01-15T19:10:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:10:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.487 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:10:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.488 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:10:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:10:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:10:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:10:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:10:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:10:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:10:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:10:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:10:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:10:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:10:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:10:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:10:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:10:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:10:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:10:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:10:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:10:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:10:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.034283676999998, 2021-01-15T19:10:49.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:10:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.501 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:10:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:10:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:10:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:10:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:10:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:10:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:10:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:10:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:10:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:10:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:10:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:10:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:10:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:10:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:10:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:10:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:10:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:10:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:10:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:10:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:10:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:10:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.034172531000003, 2021-01-15T19:10:49.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:10:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:10:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:10:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:10:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:10:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:10:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:10:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:10:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:10:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:10:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:10:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:10:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:10:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:10:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:10:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:10:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:10:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:10:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:10:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:10:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:10:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.527 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.528 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:10:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.942398209, 2021-01-15T19:10:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:10:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.529 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:10:49.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:49.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:10:49.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:10:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:10:49.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:10:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:10:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:10:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:10:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:10:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:10:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:10:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:10:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:10:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:10:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:10:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:10:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:10:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:10:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:10:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:10:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:10:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:10:49.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:10:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:10:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.941115069999999, 2021-01-15T19:10:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:10:49.544 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:49.545 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:10:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:49.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:49.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:10:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:10:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:49.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:10:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:10:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:49.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:10:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:10:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:49.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:10:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:10:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:49.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:10:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:10:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:10:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:10:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:10:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:10:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:10:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:10:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:10:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:10:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:10:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:10:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:10:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:10:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:49.559 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:49.562 INFO:tasks.rados.rados.0.smithi001.stdout:773: writing smithi00140323-22 from 3817472 to 4546560 tid 3 2021-01-15T19:10:49.563 INFO:tasks.rados.rados.0.smithi001.stdout:773: writing smithi00140323-22 from 4546560 to 4734976 tid 4 2021-01-15T19:10:49.563 INFO:tasks.rados.rados.0.smithi001.stdout:771: expect (ObjNum 270 snap 81 seq_num 270) 2021-01-15T19:10:49.563 INFO:tasks.rados.rados.0.smithi001.stdout:772: finishing copy_from to smithi00140323-45 2021-01-15T19:10:49.563 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 101 (ObjNum 256 snap 79 seq_num 256) dirty exists 2021-01-15T19:10:49.563 INFO:tasks.rados.rados.0.smithi001.stdout:772: finishing copy_from racing read to smithi00140323-45 2021-01-15T19:10:49.564 INFO:tasks.rados.rados.0.smithi001.stdout:773: finishing write tid 1 to smithi00140323-22 2021-01-15T19:10:49.564 INFO:tasks.rados.rados.0.smithi001.stdout:773: finishing write tid 2 to smithi00140323-22 2021-01-15T19:10:49.564 INFO:tasks.rados.rados.0.smithi001.stdout:773: finishing write tid 3 to smithi00140323-22 2021-01-15T19:10:49.564 INFO:tasks.rados.rados.0.smithi001.stdout:773: finishing write tid 4 to smithi00140323-22 2021-01-15T19:10:49.565 INFO:tasks.rados.rados.0.smithi001.stdout:773: finishing write tid 5 to smithi00140323-22 2021-01-15T19:10:49.565 INFO:tasks.rados.rados.0.smithi001.stdout:773: finishing write tid 6 to smithi00140323-22 2021-01-15T19:10:49.565 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 22 v 197 (ObjNum 272 snap 83 seq_num 272) dirty exists 2021-01-15T19:10:49.565 INFO:tasks.rados.rados.0.smithi001.stdout:773: left oid 22 (ObjNum 272 snap 83 seq_num 272) 2021-01-15T19:10:49.565 INFO:tasks.rados.rados.0.smithi001.stdout:771: done (4 left) 2021-01-15T19:10:49.566 INFO:tasks.rados.rados.0.smithi001.stdout:772: done (3 left) 2021-01-15T19:10:49.566 INFO:tasks.rados.rados.0.smithi001.stdout:773: done (2 left) 2021-01-15T19:10:49.566 INFO:tasks.rados.rados.0.smithi001.stdout:774: read oid 18 snap 74 2021-01-15T19:10:49.566 INFO:tasks.rados.rados.0.smithi001.stdout:774: expect (ObjNum 228 snap 71 seq_num 228) 2021-01-15T19:10:49.566 INFO:tasks.rados.rados.0.smithi001.stdout:775: copy_from oid 42 from oid 46 current snap is 83 2021-01-15T19:10:49.567 INFO:tasks.rados.rados.0.smithi001.stdout:776: read oid 41 snap -1 2021-01-15T19:10:49.567 INFO:tasks.rados.rados.0.smithi001.stdout:776: expect (ObjNum 241 snap 75 seq_num 241) 2021-01-15T19:10:49.567 INFO:tasks.rados.rados.0.smithi001.stdout:777: read oid 20 snap -1 2021-01-15T19:10:49.567 INFO:tasks.rados.rados.0.smithi001.stdout:777: expect (ObjNum 19 snap 0 seq_num 19) 2021-01-15T19:10:49.567 INFO:tasks.rados.rados.0.smithi001.stdout:778: read oid 34 snap 68 2021-01-15T19:10:49.568 INFO:tasks.rados.rados.0.smithi001.stdout:778: expect (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:10:49.568 INFO:tasks.rados.rados.0.smithi001.stdout:779: copy_from oid 19 from oid 5 current snap is 83 2021-01-15T19:10:49.568 INFO:tasks.rados.rados.0.smithi001.stdout:780: rollback oid 47 current snap is 83 2021-01-15T19:10:49.568 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 47 to 72 2021-01-15T19:10:49.568 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 35 current snap is 83 2021-01-15T19:10:49.569 INFO:tasks.rados.rados.0.smithi001.stdout:781: seq_num 273 ranges {1794048=581632,2375680=565248,2940928=532480,3473408=507904} 2021-01-15T19:10:49.569 INFO:tasks.rados.rados.0.smithi001.stdout:781: writing smithi00140323-35 from 1794048 to 2375680 tid 1 2021-01-15T19:10:49.569 INFO:tasks.rados.rados.0.smithi001.stdout:781: writing smithi00140323-35 from 2375680 to 2940928 tid 2 2021-01-15T19:10:49.569 INFO:tasks.rados.rados.0.smithi001.stdout:781: writing smithi00140323-35 from 2940928 to 3473408 tid 3 2021-01-15T19:10:49.569 INFO:tasks.rados.rados.0.smithi001.stdout:781: writing smithi00140323-35 from 3473408 to 3981312 tid 4 2021-01-15T19:10:49.570 INFO:tasks.rados.rados.0.smithi001.stdout:782: read oid 24 snap 55 2021-01-15T19:10:49.570 INFO:tasks.rados.rados.0.smithi001.stdout:782: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:10:49.570 INFO:tasks.rados.rados.0.smithi001.stdout:783: read oid 8 snap -1 2021-01-15T19:10:49.570 INFO:tasks.rados.rados.0.smithi001.stdout:783: expect (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:49.571 INFO:tasks.rados.rados.0.smithi001.stdout:780: finishing rollback tid 0 to smithi00140323-47 2021-01-15T19:10:49.571 INFO:tasks.rados.rados.0.smithi001.stdout:780: finishing rollback tid 1 to smithi00140323-47 2021-01-15T19:10:49.571 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 47 v 200 (ObjNum 0 snap 0 seq_num 0) dirty dne 2021-01-15T19:10:49.571 INFO:tasks.rados.rados.0.smithi001.stdout:774: expect (ObjNum 228 snap 71 seq_num 228) 2021-01-15T19:10:49.571 INFO:tasks.rados.rados.0.smithi001.stdout:776: expect (ObjNum 241 snap 75 seq_num 241) 2021-01-15T19:10:49.572 INFO:tasks.rados.rados.0.smithi001.stdout:781: finishing write tid 1 to smithi00140323-35 2021-01-15T19:10:49.572 INFO:tasks.rados.rados.0.smithi001.stdout:781: finishing write tid 2 to smithi00140323-35 2021-01-15T19:10:49.572 INFO:tasks.rados.rados.0.smithi001.stdout:781: finishing write tid 3 to smithi00140323-35 2021-01-15T19:10:49.572 INFO:tasks.rados.rados.0.smithi001.stdout:781: finishing write tid 4 to smithi00140323-35 2021-01-15T19:10:49.572 INFO:tasks.rados.rados.0.smithi001.stdout:781: finishing write tid 5 to smithi00140323-35 2021-01-15T19:10:49.573 INFO:tasks.rados.rados.0.smithi001.stdout:781: finishing write tid 6 to smithi00140323-35 2021-01-15T19:10:49.573 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 35 v 205 (ObjNum 273 snap 83 seq_num 273) dirty exists 2021-01-15T19:10:49.573 INFO:tasks.rados.rados.0.smithi001.stdout:781: left oid 35 (ObjNum 273 snap 83 seq_num 273) 2021-01-15T19:10:49.573 INFO:tasks.rados.rados.0.smithi001.stdout:778: expect (ObjNum 135 snap 32 seq_num 135) 2021-01-15T19:10:49.574 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:49.574 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:49.559+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:49.661 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:49.661+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:49.697 INFO:tasks.rados.rados.0.smithi001.stdout:782: expect (ObjNum 78 snap 13 seq_num 78) 2021-01-15T19:10:49.733 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:49.733 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:49.734 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:49.734 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:49.743 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:49.784 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:49.783+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:49.863 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:49.862+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:49.908 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:49.908 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:49.909 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:49.909 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:49.909 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:49.910 INFO:tasks.rados.rados.0.smithi001.stdout:774: done (11 left) 2021-01-15T19:10:49.910 INFO:tasks.rados.rados.0.smithi001.stdout:776: done (10 left) 2021-01-15T19:10:49.910 INFO:tasks.rados.rados.0.smithi001.stdout:778: done (9 left) 2021-01-15T19:10:49.910 INFO:tasks.rados.rados.0.smithi001.stdout:780: done (8 left) 2021-01-15T19:10:49.911 INFO:tasks.rados.rados.0.smithi001.stdout:781: done (7 left) 2021-01-15T19:10:49.911 INFO:tasks.rados.rados.0.smithi001.stdout:782: done (6 left) 2021-01-15T19:10:49.911 INFO:tasks.rados.rados.0.smithi001.stdout:784: rollback oid 4 current snap is 83 2021-01-15T19:10:49.911 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 4 to 81 2021-01-15T19:10:49.912 INFO:tasks.rados.rados.0.smithi001.stdout:785: read oid 26 snap -1 2021-01-15T19:10:49.912 INFO:tasks.rados.rados.0.smithi001.stdout:785: expect deleted 2021-01-15T19:10:49.912 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 16 current snap is 83 2021-01-15T19:10:49.912 INFO:tasks.rados.rados.0.smithi001.stdout:786: seq_num 274 ranges {3620864=598016,4218880=499712,4718592=745472,5464064=360448} 2021-01-15T19:10:49.922 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:49.932 INFO:tasks.rados.rados.0.smithi001.stdout:786: writing smithi00140323-16 from 3620864 to 4218880 tid 1 2021-01-15T19:10:49.936 INFO:tasks.rados.rados.0.smithi001.stdout:786: writing smithi00140323-16 from 4218880 to 4718592 tid 2 2021-01-15T19:10:49.941 INFO:tasks.rados.rados.0.smithi001.stdout:786: writing smithi00140323-16 from 4718592 to 5464064 tid 3 2021-01-15T19:10:49.944 INFO:tasks.rados.rados.0.smithi001.stdout:786: writing smithi00140323-16 from 5464064 to 5824512 tid 4 2021-01-15T19:10:49.944 INFO:tasks.rados.rados.0.smithi001.stdout:787: rollback oid 45 current snap is 83 2021-01-15T19:10:49.944 INFO:tasks.rados.rados.0.smithi001.stdout:rollback oid 45 to 77 2021-01-15T19:10:49.945 INFO:tasks.rados.rados.0.smithi001.stdout:783: expect (ObjNum 217 snap 68 seq_num 217) 2021-01-15T19:10:49.964 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:49.962+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:49.992 INFO:teuthology.orchestra.run.smithi001.stdout: marking collection for removal 2021-01-15T19:10:49.992 INFO:teuthology.orchestra.run.smithi001.stdout:setting '_remove' omap key 2021-01-15T19:10:49.992 INFO:teuthology.orchestra.run.smithi001.stdout:finish_remove_pgs 3.19s2_head removing 3.19s2 2021-01-15T19:10:50.015 INFO:teuthology.orchestra.run.smithi001.stdout:Remove successful 2021-01-15T19:10:50.065 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:50.064+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:50.089 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:50.089 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:50.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:50.090 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:50.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.091 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:10:50.091 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:50.091 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.80066053100001, 2021-01-15T19:10:50.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:10:50.092 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.092 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.092 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:10:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:10:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:10:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:10:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:10:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:10:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:10:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:10:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:10:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:10:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:10:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:10:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:10:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:10:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:10:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:10:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:10:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:10:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:10:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:10:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:10:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:10:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:10:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:10:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:10:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:10:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.86889776500001, 2021-01-15T19:10:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:10:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:10:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:10:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:10:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:10:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:10:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:10:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:10:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:10:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:10:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:10:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:10:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:10:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:10:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:10:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:10:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:10:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:10:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:10:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:10:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:10:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:10:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:10:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:10:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:10:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:10:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:10:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.866609724, 2021-01-15T19:10:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:10:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:10:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:10:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:10:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:10:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:10:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:10:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:10:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:10:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:10:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:10:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:10:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:10:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:10:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:10:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:10:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:10:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:10:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:10:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:10:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:10:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:10:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:10:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.86402009899999, 2021-01-15T19:10:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:10:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:10:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:10:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:10:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:10:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:10:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:10:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:10:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:10:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:10:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:10:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:10:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:10:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:10:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:10:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:10:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:10:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:10:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:10:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:10:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:10:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:10:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:10:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:10:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.86311182099999, 2021-01-15T19:10:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:10:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:10:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:10:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:10:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:10:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:10:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:10:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:10:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:10:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:10:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:10:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:10:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:10:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:10:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:10:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:10:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:10:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:10:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:10:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:10:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:10:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:10:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.894427914, 2021-01-15T19:10:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:10:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:10:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:10:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:10:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:10:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:10:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:10:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:10:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:10:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:10:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:10:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:10:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:10:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:10:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:10:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:10:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:10:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:10:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:10:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:10:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:10:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:10:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:10:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:10:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:10:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:10:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:10:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:10:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.89427817699999, 2021-01-15T19:10:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:10:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:10:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:10:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:10:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:10:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:10:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:10:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:10:50.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:10:50.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:10:50.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:10:50.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:10:50.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:10:50.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:50.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:10:50.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:10:50.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:10:50.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:10:50.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:50.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:10:50.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:10:50.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:10:50.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:10:50.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:10:50.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:10:50.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:10:50.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:10:50.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:10:50.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:10:50.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:10:50.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:10:50.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.205 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.205 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.206 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:10:50.206 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:50.206 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.89412627900001, 2021-01-15T19:10:50.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:10:50.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.208 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:10:50.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:50.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:50.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:10:50.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:10:50.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:10:50.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:10:50.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:10:50.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:10:50.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:10:50.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:10:50.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:10:50.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:10:50.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:10:50.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:50.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:10:50.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:10:50.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:10:50.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:10:50.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:10:50.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:10:50.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:10:50.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:10:50.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:10:50.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:10:50.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:10:50.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:10:50.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:10:50.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:10:50.222 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.223 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.223 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:10:50.224 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:50.224 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.89400096700001, 2021-01-15T19:10:50.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:10:50.224 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.224 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.225 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:10:50.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:50.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:10:50.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:10:50.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:10:50.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:10:50.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:10:50.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:10:50.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:10:50.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:10:50.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:10:50.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:10:50.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:10:50.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:50.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:10:50.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:10:50.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:10:50.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:10:50.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:50.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:10:50.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:10:50.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:10:50.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:10:50.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:10:50.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:10:50.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:10:50.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:10:50.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:10:50.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:10:50.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:10:50.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:10:50.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.241 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.241 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.242 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:10:50.242 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:50.242 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.873168392, 2021-01-15T19:10:50.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:10:50.243 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.243 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.243 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.244 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:10:50.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.244 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:50.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:50.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:10:50.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:10:50.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:10:50.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:10:50.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:10:50.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:10:50.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:10:50.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:10:50.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:10:50.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:10:50.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:10:50.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:10:50.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:10:50.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:10:50.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:10:50.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:10:50.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:10:50.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:10:50.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:10:50.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:10:50.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:10:50.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.256 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.257 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:10:50.257 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:50.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.86687506000001, 2021-01-15T19:10:50.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:10:50.258 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.258 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.259 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:10:50.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:50.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:50.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:10:50.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:10:50.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:10:50.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:10:50.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:10:50.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:10:50.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:10:50.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:10:50.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:10:50.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:10:50.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:10:50.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:10:50.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:10:50.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:10:50.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:10:50.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:10:50.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:10:50.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:10:50.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:10:50.270 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.270 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.270 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.271 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:10:50.271 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:50.271 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.904815547, 2021-01-15T19:10:50.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:10:50.272 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.272 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.272 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.272 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.272 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.273 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:10:50.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.273 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:50.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:50.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:10:50.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:10:50.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:10:50.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:10:50.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:10:50.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:10:50.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:10:50.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:10:50.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:10:50.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:10:50.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:10:50.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:50.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:10:50.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:10:50.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:10:50.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:10:50.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:10:50.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:10:50.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:10:50.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:10:50.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:10:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:10:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:10:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:10:50.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:10:50.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:10:50.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.287 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.287 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.288 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:50.288 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:50.288 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.842443567, 2021-01-15T19:10:50.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:10:50.288 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.289 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.289 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:10:50.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.290 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:50.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:50.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:10:50.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:10:50.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:10:50.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:10:50.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:10:50.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:10:50.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:10:50.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:10:50.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:10:50.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:10:50.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:10:50.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:10:50.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:10:50.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:10:50.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:10:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:10:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:10:50.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:10:50.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:10:50.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:10:50.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:10:50.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.302 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.303 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:50.303 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:50.303 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.839141066, 2021-01-15T19:10:50.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:10:50.303 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.304 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.304 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.304 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.304 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.304 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:10:50.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:50.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:50.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:10:50.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:10:50.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:50.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:10:50.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:10:50.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:10:50.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:10:50.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:10:50.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:10:50.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:10:50.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:10:50.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:10:50.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:10:50.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:10:50.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:10:50.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:10:50.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:10:50.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:10:50.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:10:50.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:10:50.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:10:50.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:50.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:50.318 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.83781920000001, 2021-01-15T19:10:50.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:10:50.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.319 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.319 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:10:50.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:50.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:50.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:10:50.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:10:50.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:10:50.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:10:50.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:10:50.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:10:50.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:10:50.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:10:50.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:10:50.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:10:50.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:10:50.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:10:50.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:10:50.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:10:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:10:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:10:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:10:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:10:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:10:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:10:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:10:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.83744894700001, 2021-01-15T19:10:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:10:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:10:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:10:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:10:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:10:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:10:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:10:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:10:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:10:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:10:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:10:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:10:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:10:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:10:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:10:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:10:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:10:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:10:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:10:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:10:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:10:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:10:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:10:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.83711455400001, 2021-01-15T19:10:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:10:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:10:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:10:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:10:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:10:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:10:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:10:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:10:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:10:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:10:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:10:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:10:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:10:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:10:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:10:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:10:50.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:10:50.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:10:50.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:10:50.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:10:50.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:10:50.360 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.360 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.361 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:10:50.361 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:50.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 116.89585329499999, 2021-01-15T19:10:50.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:10:50.362 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.362 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:50.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:50.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:10:50.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:10:50.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:10:50.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:10:50.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:10:50.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:10:50.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:10:50.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:10:50.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:10:50.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:10:50.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:10:50.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:50.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:10:50.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:10:50.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:10:50.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:10:50.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:10:50.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:10:50.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:10:50.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:10:50.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.374 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.374 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:10:50.375 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:50.375 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.90349042299999, 2021-01-15T19:10:50.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:10:50.375 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.375 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.376 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.376 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:10:50.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.377 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:50.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:50.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:10:50.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:10:50.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:10:50.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:10:50.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:10:50.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:10:50.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:50.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:10:50.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:10:50.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:10:50.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:10:50.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:50.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:10:50.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:10:50.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:10:50.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:10:50.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:10:50.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:10:50.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:10:50.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:10:50.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:10:50.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:10:50.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:10:50.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:10:50.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:10:50.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:10:50.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.391 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.392 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:10:50.392 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:50.392 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.873496219, 2021-01-15T19:10:50.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:10:50.393 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.393 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.393 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.393 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.393 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.394 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:10:50.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.394 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:50.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:50.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:10:50.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:10:50.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:10:50.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:10:50.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:10:50.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:10:50.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:10:50.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:10:50.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:10:50.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:10:50.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:10:50.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:10:50.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:10:50.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:10:50.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:10:50.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:10:50.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:10:50.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:10:50.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:10:50.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.406 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:50.408 INFO:tasks.rados.rados.0.smithi001.stdout:783: done (9 left) 2021-01-15T19:10:50.409 INFO:tasks.rados.rados.0.smithi001.stdout:785: done (8 left) 2021-01-15T19:10:50.409 INFO:tasks.rados.rados.0.smithi001.stdout:append oid 11 current snap is 83 2021-01-15T19:10:50.409 INFO:tasks.rados.rados.0.smithi001.stdout:787: finishing rollback tid 0 to smithi00140323-45 2021-01-15T19:10:50.409 INFO:tasks.rados.rados.0.smithi001.stdout:787: finishing rollback tid 1 to smithi00140323-45 2021-01-15T19:10:50.409 INFO:tasks.rados.rados.0.smithi001.stdout:787: finishing rollback tid 2 to smithi00140323-45 2021-01-15T19:10:50.410 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 45 v 396 (ObjNum 89 snap 16 seq_num 89) dirty exists 2021-01-15T19:10:50.410 INFO:tasks.rados.rados.0.smithi001.stdout:786: finishing write tid 1 to smithi00140323-16 2021-01-15T19:10:50.410 INFO:tasks.rados.rados.0.smithi001.stdout:786: finishing write tid 2 to smithi00140323-16 2021-01-15T19:10:50.410 INFO:tasks.rados.rados.0.smithi001.stdout:786: finishing write tid 3 to smithi00140323-16 2021-01-15T19:10:50.410 INFO:tasks.rados.rados.0.smithi001.stdout:786: finishing write tid 4 to smithi00140323-16 2021-01-15T19:10:50.411 INFO:tasks.rados.rados.0.smithi001.stdout:788: seq_num 275 ranges {0=614400,614400=786432,1400832=499712,1900544=319488} 2021-01-15T19:10:50.411 INFO:tasks.rados.rados.0.smithi001.stdout:788: writing smithi00140323-11 from 0 to 614400 tid 1 2021-01-15T19:10:50.411 INFO:tasks.rados.rados.0.smithi001.stdout:788: writing smithi00140323-11 from 614400 to 1400832 tid 2 2021-01-15T19:10:50.411 INFO:tasks.rados.rados.0.smithi001.stdout:788: writing smithi00140323-11 from 1400832 to 1900544 tid 3 2021-01-15T19:10:50.411 INFO:tasks.rados.rados.0.smithi001.stdout:788: writing smithi00140323-11 from 1900544 to 2220032 tid 4 2021-01-15T19:10:50.412 INFO:tasks.rados.rados.0.smithi001.stdout:787: done (8 left) 2021-01-15T19:10:50.412 INFO:tasks.rados.rados.0.smithi001.stdout:789: read oid 13 snap -1 2021-01-15T19:10:50.412 INFO:tasks.rados.rados.0.smithi001.stdout:789: expect (ObjNum 261 snap 79 seq_num 261) 2021-01-15T19:10:50.412 INFO:tasks.rados.rados.0.smithi001.stdout:790: delete oid 17 current snap is 83 2021-01-15T19:10:50.412 INFO:tasks.rados.rados.0.smithi001.stdout:786: finishing write tid 5 to smithi00140323-16 2021-01-15T19:10:50.413 INFO:tasks.rados.rados.0.smithi001.stdout:786: finishing write tid 6 to smithi00140323-16 2021-01-15T19:10:50.413 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 16 v 206 (ObjNum 274 snap 83 seq_num 274) dirty exists 2021-01-15T19:10:50.413 INFO:tasks.rados.rados.0.smithi001.stdout:786: left oid 16 (ObjNum 274 snap 83 seq_num 274) 2021-01-15T19:10:50.413 INFO:tasks.rados.rados.0.smithi001.stdout:788: finishing write tid 1 to smithi00140323-11 2021-01-15T19:10:50.413 INFO:tasks.rados.rados.0.smithi001.stdout:788: finishing write tid 2 to smithi00140323-11 2021-01-15T19:10:50.414 INFO:tasks.rados.rados.0.smithi001.stdout:788: finishing write tid 3 to smithi00140323-11 2021-01-15T19:10:50.414 INFO:tasks.rados.rados.0.smithi001.stdout:788: finishing write tid 4 to smithi00140323-11 2021-01-15T19:10:50.414 INFO:tasks.rados.rados.0.smithi001.stdout:788: finishing write tid 5 to smithi00140323-11 2021-01-15T19:10:50.414 INFO:tasks.rados.rados.0.smithi001.stdout:788: finishing write tid 6 to smithi00140323-11 2021-01-15T19:10:50.414 INFO:tasks.rados.rados.0.smithi001.stdout:update_object_version oid 11 v 402 (ObjNum 275 snap 83 seq_num 275) dirty exists 2021-01-15T19:10:50.415 INFO:tasks.rados.rados.0.smithi001.stdout:788: left oid 11 (ObjNum 275 snap 83 seq_num 275) 2021-01-15T19:10:50.415 INFO:tasks.rados.rados.0.smithi001.stdout:789: expect (ObjNum 261 snap 79 seq_num 261) 2021-01-15T19:10:50.415 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:50.416 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:50.406+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:50.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2021-01-15T19:10:50.508 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:50.507+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:50.575 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:50.575 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:50.575 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:50.576 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:50.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:50.608 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:50.607+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:50.709 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:50.708+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:50.759 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:50.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:50.843 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:50.842+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:50.911 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:50.910+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:50.928 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:50.928 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:50.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:50.928 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:50.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:10:50.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:50.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 131.74108157500001, 2021-01-15T19:10:50.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:10:50.930 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:50.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:50.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:50.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:10:50.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:10:50.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:10:50.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:10:50.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:10:50.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:10:50.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:10:50.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:10:50.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:10:50.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:10:50.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:10:50.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:10:50.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:10:50.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:10:50.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:10:50.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:10:50.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:10:50.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.940 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:50.941 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:50.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.693593398, 2021-01-15T19:10:50.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:10:50.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:50.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:10:50.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:50.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:50.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:10:50.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:10:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:10:50.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:10:50.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:10:50.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:10:50.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:50.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:10:50.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:10:50.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:10:50.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:10:50.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:10:50.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:10:50.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:50.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:10:50.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:10:50.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:50.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:10:50.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:10:50.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:50.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:10:50.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:10:50.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:10:50.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:10:50.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:10:50.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:50.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 127.69344538, 2021-01-15T19:10:50.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:10:50.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.958 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.958 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:10:50.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.959 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:50.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:50.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:10:50.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:10:50.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:10:50.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:10:50.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:10:50.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:10:50.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:10:50.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:10:50.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:10:50.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:10:50.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:10:50.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:50.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:10:50.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:10:50.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:10:50.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:10:50.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:50.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:10:50.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:10:50.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:10:50.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:10:50.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:10:50.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:10:50.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:10:50.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:10:50.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.973 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:50.973 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:50.973 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.739124388999997, 2021-01-15T19:10:50.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:10:50.974 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.974 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.975 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:50.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:50.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:50.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:10:50.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:10:50.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:10:50.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:10:50.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:10:50.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:10:50.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:10:50.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:10:50.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:10:50.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:10:50.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:10:50.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:50.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:10:50.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:10:50.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:10:50.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:10:50.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:50.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:10:50.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:10:50.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:50.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:10:50.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:10:50.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:50.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:50.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:50.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:50.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.738933099999997, 2021-01-15T19:10:50.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:10:50.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:50.989 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:50.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:50.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:50.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:50.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:50.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.990 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:50.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:50.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:50.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:50.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:50.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:50.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:10:50.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:50.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:10:50.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:10:50.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:50.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:10:50.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:10:50.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:50.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:10:50.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:10:50.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:50.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:10:50.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:10:50.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:10:50.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:10:50.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:50.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:10:50.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:10:50.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:50.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:10:50.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:10:50.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:50.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:50.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:10:51.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:10:51.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:10:51.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:10:51.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.001 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.002 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:51.002 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:51.002 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.738770747000004, 2021-01-15T19:10:51.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:10:51.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.003 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.004 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.004 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:51.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:51.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:51.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:10:51.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:10:51.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:10:51.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:10:51.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:10:51.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:10:51.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:10:51.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:10:51.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:10:51.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:10:51.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:10:51.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:51.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:10:51.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:10:51.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:10:51.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:10:51.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:10:51.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:10:51.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:10:51.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:10:51.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.016 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:51.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:51.017 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.707050834, 2021-01-15T19:10:51.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:10:51.017 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.018 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.018 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:10:51.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.019 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:51.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:51.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:10:51.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:10:51.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:10:51.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:10:51.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:10:51.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:10:51.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:10:51.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:10:51.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:10:51.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:10:51.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:10:51.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:10:51.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:10:51.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:10:51.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:10:51.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.031 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.032 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:51.032 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:51.032 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.706989144999994, 2021-01-15T19:10:51.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:10:51.033 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.033 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.033 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.033 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.033 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.034 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:10:51.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.034 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:51.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:51.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:10:51.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:10:51.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:10:51.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:10:51.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:10:51.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:10:51.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:10:51.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:10:51.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:10:51.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:10:51.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:10:51.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:10:51.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:10:51.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:10:51.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:10:51.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.043 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.044 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:10:51.044 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:51.045 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.602922806999999, 2021-01-15T19:10:51.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:10:51.045 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.045 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:51.045 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.046 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:10:51.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.046 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:51.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:51.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:10:51.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:10:51.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:10:51.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:10:51.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:10:51.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:10:51.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:10:51.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:10:51.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:10:51.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:51.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:10:51.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:10:51.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:10:51.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:10:51.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:10:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:10:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:10:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:10:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:10:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:10:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:10:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:10:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:10:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:10:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.060 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:10:51.061 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.602836234999998, 2021-01-15T19:10:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:10:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:10:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.063 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:10:51.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:10:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:10:51.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:10:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:10:51.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:10:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:10:51.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:10:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:10:51.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:10:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:10:51.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:10:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:10:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:10:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:10:51.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:10:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:10:51.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:10:51.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:10:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:10:51.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:10:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:10:51.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:10:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:10:51.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:10:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:10:51.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:10:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:10:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.712526105999999, 2021-01-15T19:10:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:10:51.080 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:10:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:10:51.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:10:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:10:51.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:10:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:10:51.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:10:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:10:51.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:10:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:10:51.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:10:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:10:51.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:10:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:10:51.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:10:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:10:51.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:10:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:10:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.587365257000002, 2021-01-15T19:10:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:10:51.092 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.093 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:10:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:10:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:10:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:10:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:10:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:10:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:10:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:10:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:10:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:10:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:10:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:10:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:10:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:10:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:10:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:10:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:10:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:10:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:10:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:10:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:10:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.106 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:10:51.107 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.587269337000002, 2021-01-15T19:10:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:10:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.108 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:10:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.109 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:10:51.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:10:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:10:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:10:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:10:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:10:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:10:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:10:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:10:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:10:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:10:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:10:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:10:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:10:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:10:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:10:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:10:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:10:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:10:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:10:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:10:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:10:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:10:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.122 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:10:51.123 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.720210997000002, 2021-01-15T19:10:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:10:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.124 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:10:51.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:51.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:51.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:10:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:10:51.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:10:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:10:51.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:10:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:10:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:10:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:10:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:10:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:10:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:10:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:51.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:10:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:10:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:10:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:10:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:10:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:10:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:51.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:10:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:10:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:51.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:10:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:10:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:51.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:10:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:10:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:10:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:10:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:10:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.720119920000002, 2021-01-15T19:10:51.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:10:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.142 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:10:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:10:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:10:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:10:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:10:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:10:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:10:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:10:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:10:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:10:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:10:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:10:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:10:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:10:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:10:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:10:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:10:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:10:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:10:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:10:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:10:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:10:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:10:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:10:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:10:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:10:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:10:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:10:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:10:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:10:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7106406649999999, 2021-01-15T19:10:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:10:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:10:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:10:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:10:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:10:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:10:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:10:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:10:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:10:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:10:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:10:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:10:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:10:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:10:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:10:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:10:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:10:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:10:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:10:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:10:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:10:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:10:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:10:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:10:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:10:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:10:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:10:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:10:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:10:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:10:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:10:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7104956410000001, 2021-01-15T19:10:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:10:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:10:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:10:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:10:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:10:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:10:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:10:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:10:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:10:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:10:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:10:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:10:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:10:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:10:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:10:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:10:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:10:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:10:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:10:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:10:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:10:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:10:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:10:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:10:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:10:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:10:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:10:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:10:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:10:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:10:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:10:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:10:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:10:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5450017680000001, 2021-01-15T19:10:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:10:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:10:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:10:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:10:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:10:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:10:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:10:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:10:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:10:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:10:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:10:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:10:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:10:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:10:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:10:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:10:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:10:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:10:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:10:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:10:51.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:10:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5449143409999999, 2021-01-15T19:10:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:10:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.208 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:10:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:10:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:10:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:10:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:10:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:10:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:10:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:10:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:10:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:10:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:10:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:10:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:10:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:10:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:10:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:10:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:10:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:10:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.218 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:10:51.219 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:10:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.5448343769999999, 2021-01-15T19:10:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:10:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:51.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:10:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:51.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:10:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:10:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:10:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:10:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:10:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:10:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:10:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:10:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:10:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:10:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:10:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:10:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:10:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:10:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:10:51.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:10:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:10:51.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:51.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:51.231 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:51.233 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:51.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:51.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:51.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:51.234 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:51.235 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:10:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:10:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:51.235 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:51.236 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:51.237 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:51.237 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:51.238 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:51.232+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:51.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:51.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:10:51.333 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:51.332+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:51.401 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:51.402 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:51.411 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:51.452 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:51.451+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:51.535 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:51.534+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:51.575 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:51.575 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:51.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:51.669 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:51.668+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:51.737 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:51.736+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:51.750 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:51.750 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:51.759 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:51.838 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:51.837+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:51.913 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:51.913 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:51.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:51.939 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:51.938+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:52.016 INFO:teuthology.orchestra.run.smithi001.stdout:3.cs0 2021-01-15T19:10:52.016 INFO:teuthology.orchestra.run.smithi001.stdout:3.3s0 2021-01-15T19:10:52.017 INFO:teuthology.orchestra.run.smithi001.stdout:3.2s2 2021-01-15T19:10:52.017 INFO:teuthology.orchestra.run.smithi001.stdout:3.es1 2021-01-15T19:10:52.017 INFO:teuthology.orchestra.run.smithi001.stdout:3.15s0 2021-01-15T19:10:52.017 INFO:teuthology.orchestra.run.smithi001.stdout:3.14s0 2021-01-15T19:10:52.017 INFO:teuthology.orchestra.run.smithi001.stdout:3.13s1 2021-01-15T19:10:52.018 INFO:teuthology.orchestra.run.smithi001.stdout:3.ds0 2021-01-15T19:10:52.018 INFO:teuthology.orchestra.run.smithi001.stdout:3.12s1 2021-01-15T19:10:52.018 INFO:teuthology.orchestra.run.smithi001.stdout:2.f 2021-01-15T19:10:52.018 INFO:teuthology.orchestra.run.smithi001.stdout:2.c 2021-01-15T19:10:52.018 INFO:teuthology.orchestra.run.smithi001.stdout:2.9 2021-01-15T19:10:52.019 INFO:teuthology.orchestra.run.smithi001.stdout:2.8 2021-01-15T19:10:52.019 INFO:teuthology.orchestra.run.smithi001.stdout:2.7 2021-01-15T19:10:52.019 INFO:teuthology.orchestra.run.smithi001.stdout:1.f 2021-01-15T19:10:52.019 INFO:teuthology.orchestra.run.smithi001.stdout:3.fs0 2021-01-15T19:10:52.019 INFO:teuthology.orchestra.run.smithi001.stdout:2.6 2021-01-15T19:10:52.020 INFO:teuthology.orchestra.run.smithi001.stdout:1.c 2021-01-15T19:10:52.020 INFO:teuthology.orchestra.run.smithi001.stdout:1.7 2021-01-15T19:10:52.020 INFO:teuthology.orchestra.run.smithi001.stdout:1.4 2021-01-15T19:10:52.020 INFO:teuthology.orchestra.run.smithi001.stdout:1.2 2021-01-15T19:10:52.020 INFO:teuthology.orchestra.run.smithi001.stdout:3.17s1 2021-01-15T19:10:52.021 INFO:teuthology.orchestra.run.smithi001.stdout:1.14 2021-01-15T19:10:52.021 INFO:teuthology.orchestra.run.smithi001.stdout:3.bs2 2021-01-15T19:10:52.021 INFO:teuthology.orchestra.run.smithi001.stdout:2.a 2021-01-15T19:10:52.021 INFO:teuthology.orchestra.run.smithi001.stdout:1.12 2021-01-15T19:10:52.022 INFO:teuthology.orchestra.run.smithi001.stdout:3.18s1 2021-01-15T19:10:52.022 INFO:teuthology.orchestra.run.smithi001.stdout:2.11 2021-01-15T19:10:52.022 INFO:teuthology.orchestra.run.smithi001.stdout:1.10 2021-01-15T19:10:52.039 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:52.038+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:52.059 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:52.059 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:52.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:52.141 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:52.140+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:52.241 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:52.242 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:52.242 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:52.241+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:52.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:52.294 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:10:52.350 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:52.349+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:52.394 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:52.395 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:52.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:52.444 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:52.443+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:52.536 INFO:tasks.thrashosds.thrasher:Moving pg 3.19s2 from osd.1 to osd.0 2021-01-15T19:10:52.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.19s2.1 2021-01-15T19:10:52.545 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:52.544+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:52.578 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:52.579 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:52.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:52.675 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:52.674+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:52.747 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:52.746+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:52.763 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:52.763 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:52.773 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:52.848 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:52.847+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:52.901 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:52.902 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:52.911 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:52.952 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:52.951+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:53.049 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:53.048+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:53.076 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:53.076 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:53.086 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:53.150 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:53.149+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:53.211 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:53.211 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:53.221 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:53.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:53.283 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:53.282+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:53.353 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:53.351+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:53.387 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:53.387 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:53.390 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:53.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:53.415 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:53.436 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:53.453 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:53.453+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:53.457 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:53.478 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:53.498 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:10:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:10:53.499 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:53.500 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:53.501 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:53.501 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:53.502 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:53.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:53.524 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:53.524 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:53.525 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:53.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:53.555 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:53.554+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:53.656 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:53.654+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:53.670 INFO:teuthology.orchestra.run.smithi001.stdout:get_pg_num_history pg_num_history pg_num_history(e176 pg_nums {1={11=1,58=11,154=21},2={16=8,46=18},3={21=16,52=26}} deleted_pools ) 2021-01-15T19:10:53.670 INFO:teuthology.orchestra.run.smithi001.stdout:Importing pgid 3.19s2 2021-01-15T19:10:53.670 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:9c47a2ed:::smithi00140323-38:1b# 2021-01-15T19:10:53.671 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:9c47a2ed:::smithi00140323-38:46# 2021-01-15T19:10:53.677 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:9c47a2ed:::smithi00140323-38:4b# 2021-01-15T19:10:53.683 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:9c47a2ed:::smithi00140323-38:72# 2021-01-15T19:10:53.689 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:9c47a2ed:::smithi00140323-38:7d# 2021-01-15T19:10:53.696 INFO:teuthology.orchestra.run.smithi001.stdout:Write 2#3:9c47a2ed:::smithi00140323-38:head# 2021-01-15T19:10:53.697 INFO:teuthology.orchestra.run.smithi001.stdout:snapset 7d=[]:{1b=[19],46=[46,44,41,3e,3a,38,2f,2a,26],4b=[4b],72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56],7d=[7d,7b,7a,79,78]} 2021-01-15T19:10:53.699 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:53.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:10:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:53.700 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.410980139, 2021-01-15T19:10:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:10:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:10:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:10:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:10:53.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:10:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:10:53.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:10:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:10:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:10:53.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:10:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:10:53.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:10:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:10:53.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:10:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:10:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:10:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:10:53.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:10:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:10:53.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:10:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:10:53.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:10:53.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:10:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:10:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:10:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:10:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:10:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:10:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.47921737300001, 2021-01-15T19:10:53.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:10:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:10:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:10:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:10:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:10:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:10:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:10:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:10:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:10:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:10:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:10:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:10:53.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:10:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:10:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:10:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:10:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:10:53.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:10:53.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:10:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:10:53.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:10:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:10:53.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:10:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:10:53.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:10:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:10:53.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:10:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:10:53.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.476929332, 2021-01-15T19:10:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:10:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.736 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:10:53.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:53.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:10:53.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:10:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:10:53.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:10:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:10:53.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:10:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:10:53.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:10:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:10:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:10:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:10:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:10:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:10:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:10:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:10:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:10:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:10:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:10:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:10:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:10:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:10:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:10:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:10:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.47433970700001, 2021-01-15T19:10:53.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:10:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:10:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:10:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:10:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:10:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:10:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:10:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:10:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:10:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:10:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:10:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:10:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:10:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:10:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:10:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:10:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:10:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:10:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:10:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:10:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:10:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:10:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:10:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:10:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.765 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:10:53.766 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:53.767 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.47343142899999, 2021-01-15T19:10:53.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:10:53.767 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.767 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:53.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.768 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:10:53.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.768 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:53.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:10:53.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:10:53.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:10:53.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:10:53.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:10:53.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:10:53.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:10:53.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:10:53.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:10:53.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:10:53.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:10:53.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:10:53.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:53.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:10:53.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:10:53.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:10:53.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:10:53.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:10:53.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:10:53.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:10:53.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:10:53.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.780 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.780 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:10:53.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:53.781 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.504747522, 2021-01-15T19:10:53.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:10:53.781 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.781 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.781 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.782 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:10:53.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:53.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:10:53.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:10:53.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:10:53.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:10:53.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:10:53.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:10:53.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:10:53.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:10:53.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:10:53.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:10:53.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:10:53.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:10:53.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:53.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:10:53.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:10:53.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:10:53.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:10:53.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:10:53.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:10:53.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:10:53.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:10:53.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:10:53.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:10:53.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:10:53.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:10:53.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:10:53.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:10:53.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.803 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.804 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:10:53.804 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:53.804 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.50459778499999, 2021-01-15T19:10:53.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:10:53.804 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.805 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:53.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.805 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:10:53.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.806 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:53.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:10:53.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:10:53.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:10:53.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:10:53.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:10:53.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:10:53.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:10:53.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:10:53.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:10:53.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:10:53.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:10:53.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:10:53.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:53.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:10:53.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:10:53.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:10:53.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:10:53.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:53.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:10:53.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:10:53.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:10:53.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:10:53.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:53.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:10:53.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:10:53.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:10:53.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:10:53.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:10:53.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:10:53.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:10:53.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:10:53.821 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.821 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.821 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.822 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:10:53.822 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:53.822 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.504445887, 2021-01-15T19:10:53.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:10:53.823 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.823 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.824 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:10:53.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.824 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:53.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:10:53.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:10:53.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:10:53.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:10:53.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:10:53.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:10:53.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:10:53.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:10:53.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:10:53.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:10:53.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:10:53.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:10:53.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:53.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:10:53.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:10:53.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:10:53.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:10:53.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:10:53.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:10:53.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:10:53.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:10:53.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:10:53.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:10:53.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:10:53.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:10:53.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:10:53.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:10:53.838 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.839 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:10:53.840 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:53.840 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.50432057500001, 2021-01-15T19:10:53.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:10:53.840 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.840 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:53.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.841 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:10:53.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.841 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:53.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:10:53.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:10:53.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:10:53.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:10:53.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:10:53.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:10:53.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:10:53.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:10:53.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:10:53.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:10:53.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:10:53.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:10:53.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:53.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:10:53.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:10:53.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:10:53.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:10:53.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:53.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:10:53.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:10:53.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:10:53.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:10:53.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:53.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:10:53.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:10:53.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:10:53.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:10:53.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:10:53.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:10:53.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:10:53.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:10:53.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.857 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.858 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:10:53.858 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:53.858 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 138.48348799999999, 2021-01-15T19:10:53.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:10:53.859 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.859 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.859 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.859 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:10:53.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.860 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:53.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:10:53.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:10:53.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:10:53.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:10:53.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:10:53.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:10:53.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:10:53.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:10:53.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:10:53.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:10:53.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:10:53.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:10:53.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:10:53.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:10:53.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:10:53.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:10:53.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:10:53.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:10:53.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:10:53.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:10:53.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:10:53.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:10:53.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.872 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.873 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:10:53.873 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:53.873 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 138.47719466800001, 2021-01-15T19:10:53.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:10:53.874 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.874 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:53.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.875 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:10:53.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.875 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:53.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:10:53.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:10:53.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:10:53.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:10:53.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:10:53.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:10:53.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:10:53.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:10:53.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:10:53.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:10:53.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:10:53.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:10:53.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:53.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:10:53.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:10:53.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:10:53.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:10:53.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:10:53.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:10:53.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:10:53.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:10:53.886 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.886 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.886 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.887 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:10:53.887 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:53.887 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.515135155, 2021-01-15T19:10:53.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:10:53.888 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.888 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.889 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:10:53.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.889 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:53.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:10:53.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:10:53.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:10:53.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:10:53.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:10:53.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:10:53.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:10:53.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:10:53.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:10:53.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:10:53.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:10:53.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:10:53.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:53.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:10:53.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:10:53.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:10:53.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:10:53.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:10:53.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:10:53.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:10:53.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:10:53.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:10:53.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:10:53.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:10:53.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:10:53.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:10:53.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:10:53.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.903 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.904 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:53.904 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:53.904 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.452763175, 2021-01-15T19:10:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:10:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:10:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:10:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:10:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:10:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:10:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:10:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:10:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:10:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:10:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:10:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:10:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:10:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:10:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:10:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:10:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:10:53.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:10:53.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:10:53.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:10:53.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:10:53.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:10:53.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:10:53.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:10:53.918 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.918 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:53.919 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:53.920 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.44946067399999, 2021-01-15T19:10:53.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:10:53.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.921 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:10:53.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:53.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:10:53.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:10:53.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:10:53.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:53.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:10:53.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:10:53.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:10:53.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:10:53.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:10:53.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:10:53.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:10:53.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:10:53.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:10:53.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:10:53.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:10:53.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:10:53.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:10:53.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:10:53.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:10:53.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:10:53.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:10:53.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:10:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.935 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:53.935 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:53.935 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.44813880800001, 2021-01-15T19:10:53.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:10:53.935 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.936 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:10:53.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.937 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:53.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:10:53.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:10:53.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:10:53.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:10:53.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:10:53.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:10:53.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:10:53.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:10:53.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:10:53.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:10:53.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:10:53.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:10:53.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:10:53.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:10:53.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:10:53.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:10:53.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:10:53.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:10:53.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:10:53.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:10:53.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:10:53.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:10:53.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.949 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.950 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:53.950 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:53.950 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.44776855500001, 2021-01-15T19:10:53.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:10:53.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.952 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:10:53.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:53.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:10:53.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:10:53.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:10:53.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:10:53.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:10:53.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:10:53.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:10:53.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:10:53.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:10:53.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:10:53.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:10:53.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:10:53.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:10:53.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:10:53.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:53.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:10:53.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:10:53.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:53.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:10:53.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:10:53.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:53.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:10:53.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:10:53.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:10:53.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:10:53.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.965 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:53.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:53.965 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.44743416200001, 2021-01-15T19:10:53.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:10:53.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.966 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:53.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.967 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:10:53.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:53.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:10:53.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:10:53.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:10:53.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:10:53.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:10:53.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:10:53.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:10:53.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:10:53.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:10:53.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:10:53.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:10:53.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:10:53.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:53.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:10:53.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:10:53.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:10:53.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:10:53.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:10:53.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:10:53.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:10:53.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:10:53.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:10:53.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:53.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.50617290300001, 2021-01-15T19:10:53.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:10:53.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:53.981 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:53.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:10:53.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:10:53.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:10:53.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:10:53.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:10:53.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:10:53.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:53.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:10:53.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:10:53.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:53.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:10:53.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:10:53.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:10:53.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:10:53.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:53.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:10:53.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:10:53.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:53.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:10:53.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:10:53.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:53.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:10:53.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:10:53.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:53.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:10:53.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:10:53.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:53.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:53.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:10:53.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:53.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 110.51381003100001, 2021-01-15T19:10:53.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:10:53.994 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:53.994 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:53.994 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:53.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:53.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:53.995 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:10:53.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.995 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:53.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:53.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:53.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:53.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:53.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:10:53.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:10:53.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:53.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:10:53.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:10:53.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:53.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:53.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:53.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:10:53.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:10:53.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:10:54.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:10:54.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:10:54.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:10:54.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:10:54.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:10:54.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:54.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:10:54.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:10:54.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:10:54.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:10:54.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:10:54.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:10:54.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:10:54.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:10:54.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:10:54.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:10:54.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:54.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:10:54.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:10:54.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:10:54.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:10:54.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:10:54.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:54.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 110.483815827, 2021-01-15T19:10:54.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:10:54.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:10:54.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.013 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:54.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:10:54.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:10:54.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:10:54.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:10:54.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:10:54.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:10:54.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:10:54.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:10:54.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:10:54.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:10:54.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:10:54.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:10:54.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:54.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:10:54.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:10:54.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:10:54.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:10:54.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:10:54.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:10:54.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:10:54.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:10:54.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.024 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.024 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.025 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:54.026 INFO:teuthology.orchestra.run.smithi001.stdout:write_pg epoch 155 info 3.19s2( v 155'64 (0'0,155'64] local-lis/les=57/58 n=6 ec=52/21 lis/c=57/57 les/c/f=58/58/0 sis=57) 2021-01-15T19:10:54.027 INFO:teuthology.orchestra.run.smithi001.stdout:Import successful 2021-01-15T19:10:54.027 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:54.025+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:54.028 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:54.127 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:54.125+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:54.195 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:54.196 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:54.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:54.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /var/log/ceph/exp.3.19s2.1 2021-01-15T19:10:54.246 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:54.245+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:54.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-01-15T19:10:54.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:10:54.342 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:54.340+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:54.374 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:54.375 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:54.375 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:54.383 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:54.430 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:54.429+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:54.510 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:54.511 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:10:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:10:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:54.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.323357206, 2021-01-15T19:10:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:10:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:10:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:10:54.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:10:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:10:54.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:10:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:10:54.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:10:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:10:54.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:10:54.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:10:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:10:54.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:10:54.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:10:54.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:10:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:10:54.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:10:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:10:54.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:10:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 131.27586902900001, 2021-01-15T19:10:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:10:54.526 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.527 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:10:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:10:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:10:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:10:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:10:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:10:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:10:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:10:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:10:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:10:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:10:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:10:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:10:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:10:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:10:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:10:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:10:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:10:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:10:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:10:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:10:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:10:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:10:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.540 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:10:54.541 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 131.275721011, 2021-01-15T19:10:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:10:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:10:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.543 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:10:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:10:54.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:10:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:10:54.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:10:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:10:54.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:10:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:10:54.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:10:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:10:54.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:10:54.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:10:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:10:54.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:10:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:10:54.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:10:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:10:54.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:10:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:10:54.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:10:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:10:54.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:10:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:10:54.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:10:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.321400019999999, 2021-01-15T19:10:54.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:10:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.559 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:10:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:10:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:10:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:10:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:10:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:10:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:10:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:10:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:10:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:10:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:10:54.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:10:54.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:10:54.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:10:54.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:10:54.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:10:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:10:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:10:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:10:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:10:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:10:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:54.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:54.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.321208730999999, 2021-01-15T19:10:54.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:10:54.576 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.577 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:10:54.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.578 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:10:54.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:10:54.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:10:54.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:10:54.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:10:54.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:10:54.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:10:54.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:10:54.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:10:54.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:10:54.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:10:54.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:10:54.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:54.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:10:54.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:10:54.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:10:54.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:10:54.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:10:54.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:10:54.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:10:54.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:10:54.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.595 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.595 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.595 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:10:54.596 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:54.596 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.321046378000005, 2021-01-15T19:10:54.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:10:54.596 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.596 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.597 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.597 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:10:54.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.598 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:54.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:10:54.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:10:54.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:10:54.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:10:54.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:10:54.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:10:54.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:10:54.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:10:54.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:10:54.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:10:54.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:10:54.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:10:54.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:10:54.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:10:54.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:10:54.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:10:54.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:10:54.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:10:54.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:10:54.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:10:54.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:10:54.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.612 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.612 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:54.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:54.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.289326465000002, 2021-01-15T19:10:54.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:10:54.613 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.613 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.614 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:10:54.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:54.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:10:54.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:10:54.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:10:54.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:10:54.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:10:54.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:10:54.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:10:54.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:10:54.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:10:54.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:10:54.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:10:54.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:10:54.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:10:54.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:10:54.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:10:54.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:10:54.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.624 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.625 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:10:54.625 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:54.625 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.289264775999996, 2021-01-15T19:10:54.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:10:54.625 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.627 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:10:54.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.627 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:54.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:10:54.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:10:54.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:10:54.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:10:54.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:10:54.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:10:54.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:10:54.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:10:54.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:10:54.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:10:54.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:10:54.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:10:54.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:10:54.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:10:54.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.636 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.637 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:10:54.637 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:54.637 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.185198438, 2021-01-15T19:10:54.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:10:54.638 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.638 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:54.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:10:54.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.639 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:54.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:10:54.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:10:54.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:10:54.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:10:54.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:10:54.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:10:54.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:10:54.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:10:54.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:10:54.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:10:54.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:10:54.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:10:54.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:54.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:10:54.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:10:54.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:10:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:10:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:10:54.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:10:54.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:10:54.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:10:54.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:10:54.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:10:54.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:54.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:10:54.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:10:54.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:10:54.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:10:54.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.654 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.655 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:10:54.655 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:54.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.185111866, 2021-01-15T19:10:54.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:10:54.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.656 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.656 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:10:54.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.657 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:54.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:10:54.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:10:54.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:10:54.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:10:54.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:10:54.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:10:54.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:10:54.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:10:54.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:10:54.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:10:54.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:10:54.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:10:54.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:54.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:10:54.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:10:54.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:10:54.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:10:54.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:54.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:10:54.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:10:54.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:10:54.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:10:54.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:54.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:10:54.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:10:54.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:10:54.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:10:54.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:10:54.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:10:54.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:10:54.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:10:54.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.672 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:10:54.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:54.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.294801737, 2021-01-15T19:10:54.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:10:54.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:10:54.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:54.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:10:54.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:10:54.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:10:54.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:10:54.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:10:54.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:10:54.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:10:54.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:10:54.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:10:54.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:10:54.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:10:54.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:10:54.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:10:54.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:10:54.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:10:54.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:10:54.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:10:54.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:10:54.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.686 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:10:54.686 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:54.686 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.169640888000004, 2021-01-15T19:10:54.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:10:54.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:54.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.688 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:10:54.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.689 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:54.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:10:54.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:10:54.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:10:54.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:10:54.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:10:54.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:10:54.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:10:54.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:10:54.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:10:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:10:54.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:54.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:10:54.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:10:54.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:10:54.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:10:54.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:10:54.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:10:54.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:10:54.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:54.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:10:54.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:10:54.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:10:54.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:10:54.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.703 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.704 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:10:54.704 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:54.704 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.169544967999997, 2021-01-15T19:10:54.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:10:54.705 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.705 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.706 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.706 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:10:54.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.706 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:54.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:10:54.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:10:54.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:10:54.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:10:54.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:10:54.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:10:54.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:10:54.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:10:54.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:10:54.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:10:54.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:10:54.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:10:54.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:54.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:10:54.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:10:54.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:10:54.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:10:54.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:54.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:10:54.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:10:54.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:10:54.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:10:54.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:10:54.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:10:54.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:10:54.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:10:54.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.723 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:10:54.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:54.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.302486627999997, 2021-01-15T19:10:54.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:10:54.724 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:54.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.725 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:10:54.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:54.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:10:54.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:10:54.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:10:54.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:10:54.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:10:54.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:10:54.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:10:54.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:10:54.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:10:54.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:10:54.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:10:54.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:10:54.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:54.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:10:54.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:10:54.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:10:54.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:10:54.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:10:54.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:10:54.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:10:54.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:10:54.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:10:54.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:10:54.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:54.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:10:54.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:10:54.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:10:54.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:10:54.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:10:54.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:54.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.302395550999996, 2021-01-15T19:10:54.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:10:54.744 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.744 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.745 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:10:54.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.746 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:54.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:10:54.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:10:54.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:10:54.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:10:54.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:10:54.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:10:54.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:10:54.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:10:54.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:10:54.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:10:54.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:10:54.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:10:54.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:54.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:10:54.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:10:54.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:10:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:10:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:54.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:10:54.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:10:54.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:10:54.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:10:54.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:54.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:10:54.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:10:54.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:10:54.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:10:54.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:10:54.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:10:54.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:10:54.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:10:54.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.763 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.764 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:10:54.765 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:10:54.765 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2929162959999996, 2021-01-15T19:10:54.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:10:54.765 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:54.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.767 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:10:54.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.767 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:10:54.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:10:54.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:10:54.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2927712720000004, 2021-01-15T19:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1272773989999996, 2021-01-15T19:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.127189972, 2021-01-15T19:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1271100079999998, 2021-01-15T19:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:54.850 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:10:54.850 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":184,"fsid":"fe0e1890-043f-40bf-ad11-c5f08349f418","created":"2021-01-15T19:07:43.378003+0000","modified":"2021-01-15T19:10:52.282324+0000","last_up_change":"2021-01-15T19:10:45.255653+0000","last_in_change":"2021-01-15T19:09:47.269026+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-01-15T19:07:47.545867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":21,"pg_placement_num":21,"pg_placement_num_target":21,"pg_num_target":21,"pg_num_pending":21,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"154","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-01-15T19:07:54.484638+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":18,"pg_placement_num":18,"pg_placement_num_target":18,"pg_num_target":18,"pg_num_pending":18,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"46","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2021-01-15T19:08:08.242953+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":26,"pg_num_target":26,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"52","auid":0,"snap_mode":"selfmanaged","snap_seq":152,"snap_epoch":181,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"isaprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"e5b26f89-5c99-4764-9183-f3bb27c118f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":177,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":34833},{"type":"v1","addr":"172.21.15.1:6801","nonce":34833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":34833},{"type":"v1","addr":"172.21.15.1:6803","nonce":34833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":34833},{"type":"v1","addr":"172.21.15.1:6807","nonce":34833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":34833},{"type":"v1","addr":"172.21.15.1:6805","nonce":34833}]},"public_addr":"172.21.15.1:6801/34833","cluster_addr":"172.21.15.1:6803/34833","heartbeat_back_addr":"172.21.15.1:6807/34833","heartbeat_front_addr":"172.21.15.1:6805/34833","state":["exists"]},{"osd":1,"uuid":"99c0d455-1bd6-43aa-bfec-b6bddf87a35e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":159,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":34834},{"type":"v1","addr":"172.21.15.1:6809","nonce":34834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":34834},{"type":"v1","addr":"172.21.15.1:6811","nonce":34834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":34834},{"type":"v1","addr":"172.21.15.1:6815","nonce":34834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":34834},{"type":"v1","addr":"172.21.15.1:6813","nonce":34834}]},"public_addr":"172.21.15.1:6809/34834","cluster_addr":"172.21.15.1:6811/34834","heartbeat_back_addr":"172.21.15.1:6815/34834","heartbeat_front_addr":"172.21.15.1:6813/34834","state":["exists"]},{"osd":2,"uuid":"71fe6c4b-3d73-4e07-b49d-98596ca7d9da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6817","nonce":34863},{"type":"v1","addr":"172.21.15.1:6819","nonce":34863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6821","nonce":34863},{"type":"v1","addr":"172.21.15.1:6823","nonce":34863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":34863},{"type":"v1","addr":"172.21.15.1:6831","nonce":34863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":34863},{"type":"v1","addr":"172.21.15.1:6828","nonce":34863}]},"public_addr":"172.21.15.1:6819/34863","cluster_addr":"172.21.15.1:6823/34863","heartbeat_back_addr":"172.21.15.1:6831/34863","heartbeat_front_addr":"172.21.15.1:6828/34863","state":["exists","up"]},{"osd":3,"uuid":"8a6b018a-3a48-4a01-8526-a386f62ecb36","up":1,"in":1,"weight":0.198638916015625,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":34864},{"type":"v1","addr":"172.21.15.1:6818","nonce":34864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":34864},{"type":"v1","addr":"172.21.15.1:6822","nonce":34864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6827","nonce":34864},{"type":"v1","addr":"172.21.15.1:6829","nonce":34864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":34864},{"type":"v1","addr":"172.21.15.1:6825","nonce":34864}]},"public_addr":"172.21.15.1:6818/34864","cluster_addr":"172.21.15.1:6822/34864","heartbeat_back_addr":"172.21.15.1:6829/34864","heartbeat_front_addr":"172.21.15.1:6825/34864","state":["exists","up"]},{"osd":4,"uuid":"ff089622-e642-4bd0-901c-60efdb87a549","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":34829},{"type":"v1","addr":"172.21.15.136:6801","nonce":34829}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":34829},{"type":"v1","addr":"172.21.15.136:6803","nonce":34829}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":34829},{"type":"v1","addr":"172.21.15.136:6807","nonce":34829}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":34829},{"type":"v1","addr":"172.21.15.136:6805","nonce":34829}]},"public_addr":"172.21.15.136:6801/34829","cluster_addr":"172.21.15.136:6803/34829","heartbeat_back_addr":"172.21.15.136:6807/34829","heartbeat_front_addr":"172.21.15.136:6805/34829","state":["exists","up"]},{"osd":5,"uuid":"3c57c228-dfaa-4e33-bded-27da0048d141","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":34830},{"type":"v1","addr":"172.21.15.136:6811","nonce":34830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":34830},{"type":"v1","addr":"172.21.15.136:6815","nonce":34830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":34830},{"type":"v1","addr":"172.21.15.136:6823","nonce":34830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6817","nonce":34830},{"type":"v1","addr":"172.21.15.136:6819","nonce":34830}]},"public_addr":"172.21.15.136:6811/34830","cluster_addr":"172.21.15.136:6815/34830","heartbeat_back_addr":"172.21.15.136:6823/34830","heartbeat_front_addr":"172.21.15.136:6819/34830","state":["exists","up"]},{"osd":6,"uuid":"201f889f-dac1-4a26-a285-7a0ec21c3646","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":94,"up_from":132,"up_thru":92,"down_at":95,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":48096},{"type":"v1","addr":"172.21.15.136:6810","nonce":48096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":48096},{"type":"v1","addr":"172.21.15.136:6814","nonce":48096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":48096},{"type":"v1","addr":"172.21.15.136:6822","nonce":48096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":48096},{"type":"v1","addr":"172.21.15.136:6818","nonce":48096}]},"public_addr":"172.21.15.136:6810/48096","cluster_addr":"172.21.15.136:6814/48096","heartbeat_back_addr":"172.21.15.136:6822/48096","heartbeat_front_addr":"172.21.15.136:6818/48096","state":["exists","up"]},{"osd":7,"uuid":"79ffd413-a9d3-4180-9691-d834d89c3f2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":34874},{"type":"v1","addr":"172.21.15.136:6825","nonce":34874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":34874},{"type":"v1","addr":"172.21.15.136:6827","nonce":34874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":34874},{"type":"v1","addr":"172.21.15.136:6831","nonce":34874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":34874},{"type":"v1","addr":"172.21.15.136:6829","nonce":34874}]},"public_addr":"172.21.15.136:6825/34874","cluster_addr":"172.21.15.136:6827/34874","heartbeat_back_addr":"172.21.15.136:6831/34874","heartbeat_front_addr":"172.21.15.136:6829/34874","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:49.068217+0000","dead_epoch":177},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:09:49.114373+0000","dead_epoch":159},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:10:51.570589+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:10:50.239826+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:10:50.335135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":65536,"last_purged_snaps_scrub":"2021-01-15T19:10:51.341578+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":65536,"last_purged_snaps_scrub":"2021-01-15T19:10:00.995180+0000","dead_epoch":95},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-01-15T19:10:51.328325+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[{"pgid":"1.d","osds":[2,4]}],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"},"isaprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"2","m":"1","name":"isaprofile","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":25,"length":1},{"begin":38,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":56,"length":1},{"begin":58,"length":1},{"begin":62,"length":1},{"begin":65,"length":1},{"begin":68,"length":1},{"begin":70,"length":1},{"begin":75,"length":2},{"begin":80,"length":1},{"begin":83,"length":2},{"begin":86,"length":4},{"begin":92,"length":2},{"begin":95,"length":2},{"begin":98,"length":2},{"begin":102,"length":1},{"begin":109,"length":2},{"begin":112,"length":4},{"begin":120,"length":2},{"begin":125,"length":1},{"begin":132,"length":1},{"begin":141,"length":1},{"begin":143,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-01-15T19:10:54.851 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:54.848+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:54.851 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '154', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 18, 'pg_placement_num': 18, 'pg_placement_num_target': 18, 'pg_num_target': 18, 'pg_num_pending': 18, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '46', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '52', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 152, 'snap_epoch': 181, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}] 2021-01-15T19:10:54.852 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool device_health_metrics 2021-01-15T19:10:54.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:54.950 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:54.949+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:55.016 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:55.016 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:55.016 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:55.017 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:55.025 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:55.051 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:55.049+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:55.150 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:55.150 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:55.151 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:10:55.151 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:10:55.151 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:55.153 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:55.152+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:55.160 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:55.255 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:55.253+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:55.301 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:10:55.329 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:55.330 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:55.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:55.330 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:55.331 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:55.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:55.380 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:55.379+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:55.455 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:55.455+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:55.488 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:55.489 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:55.489 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:55.489 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:55.498 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:55.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:10:55.556 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:55.556+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:55.633 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:55.633 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:55.634 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:55.634 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:55.643 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:55.657 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:55.656+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:55.692 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:55.720 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:55.748 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:55.758 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:55.757+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:55.778 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:55.778 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:55.779 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:55.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:55.800 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:55.822 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:55.822 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:10:55.823 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:10:55.823 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:55.824 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:55.824 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:10:55.825 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:10:55.825 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:10:55.825 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:10:55.826 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:10:55.826 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:10:55.827 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:55.828 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:55.868 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:55.867+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:55.921 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:55.922 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:55.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:55.962 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:55.960+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:Nothing to do for non-filestore backend 2021-01-15T19:10:56.064 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:56.062+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:56.110 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:56.111 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:56.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:56.201 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:56.200+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:56.266 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:56.265+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:56.289 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:56.290 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:56.300 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:56.367 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:56.366+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:56.436 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:56.437 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:56.446 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:10:56.490 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:56.489+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:56.549 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool rbd 2021-01-15T19:10:56.571 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:56.570+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:56.572 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:56.572 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:56.582 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:10:56.672 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:56.670+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:56.747 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:56.747 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:56.757 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:10:56.806 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:56.805+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:56.876 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:56.874+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:56.884 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:56.885 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:56.894 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:10:56.977 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:56.975+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:57.068 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:57.069 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:57.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:10:57.079 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:57.077+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:57.178 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:57.177+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:57.208 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:57.209 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:57.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:10:57.309 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:57.307+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:57.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:10:57.382 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:57.381+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:57.387 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:57.387 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:57.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:10:57.489 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:57.488+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:57.567 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:57.567 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:57.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:10:57.583 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:57.581+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:57.683 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:57.682+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:57.715 INFO:teuthology.orchestra.run.smithi001.stderr:Nothing to do for non-filestore backend 2021-01-15T19:10:57.741 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:57.741 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:57.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:10:57.784 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:57.782+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:57.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:10:57.885 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:57.884+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:57.916 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:57.916 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:57.925 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:10:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:10:57.986 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:57.985+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:10:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:10:58.039 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:10:58.051 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:10:58.051 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:10:58.061 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:10:58.061 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:10:58.081 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:10:58.081 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:10:58.081 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:10:58.081 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:10:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:10:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:10:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:10:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:10:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:10:58.083 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:10:58.083 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:10:58.083 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:10:58.084 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:10:58.108 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:58.106+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:58.186 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:58.186 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:10:58.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:10:58.187 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:10:58.187 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:58.189 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:10:58.188+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:10:58.196 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:10:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:10:58.234 DEBUG:teuthology.orchestra.run.smithi001:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool unique_pool_0 2021-01-15T19:10:58.290 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:58.289+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:58.334 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:58.334 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:58.335 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:58.335 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:58.345 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:10:58.390 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:58.389+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:58.478 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:58.479 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:58.479 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:58.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:10:58.491 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:58.489+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:58.592 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:58.591+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:58.622 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:58.622 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:58.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:58.623 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:58.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.623 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:58.623 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:58.624 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.18438047800001, 2021-01-15T19:10:58.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:10:58.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.625 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:10:58.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.625 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:58.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:10:58.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:10:58.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:10:58.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:10:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:10:58.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:10:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:10:58.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:10:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:10:58.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:10:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:10:58.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:10:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:10:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:10:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:10:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:10:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:10:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:10:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:10:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:10:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:10:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:10:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:10:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:10:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:10:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:10:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:10:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:10:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:10:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:10:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.18424117800001, 2021-01-15T19:10:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:10:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:10:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:10:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:10:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:10:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:10:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:10:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:10:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:10:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:10:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:10:58.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:10:58.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:10:58.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:10:58.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:10:58.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:10:58.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:10:58.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:10:58.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:10:58.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:10:58.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:10:58.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:10:58.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:10:58.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:10:58.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:10:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:10:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:10:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:10:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:10:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:10:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:10:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:10:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.661 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:10:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:58.662 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.18412007200001, 2021-01-15T19:10:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:10:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.663 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:10:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:10:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:10:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:10:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:10:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:10:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:10:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:10:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:10:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:10:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:10:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:10:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:10:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:10:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:10:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:10:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:10:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:10:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:10:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:10:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:10:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:10:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:10:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:10:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:10:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:10:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:10:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:10:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:10:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:10:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:10:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:58.681 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.41383919800001, 2021-01-15T19:10:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:10:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.682 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:10:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:10:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:10:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:10:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:10:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:10:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:10:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:10:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:10:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:10:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:10:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:10:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:10:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:10:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:10:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:10:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:10:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:10:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:10:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:10:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:10:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:10:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:10:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:10:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:10:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:10:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:10:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.40486102, 2021-01-15T19:10:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:10:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:10:58.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:58.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:10:58.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:10:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:10:58.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:10:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:10:58.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:10:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:10:58.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:10:58.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:10:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:10:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:10:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:10:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:10:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:10:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:10:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:10:58.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:10:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:10:58.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:10:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:10:58.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:10:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:10:58.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:10:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:10:58.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.716 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.40434131999999, 2021-01-15T19:10:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:10:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:10:58.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:58.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:10:58.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:10:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:10:58.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:10:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:10:58.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:10:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:10:58.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:10:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:10:58.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:10:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:10:58.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:10:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:10:58.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:10:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:10:58.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:10:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:10:58.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:10:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:10:58.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:10:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:10:58.730 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:10:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:10:58.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:10:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:10:58.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:10:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.40266036599999, 2021-01-15T19:10:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:10:58.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:10:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:10:58.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:10:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:10:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:10:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:10:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:10:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:10:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:10:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:10:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:10:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:10:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:10:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:10:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:10:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:10:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:10:58.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:10:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:10:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:10:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:10:58.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:10:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:10:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:10:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:10:58.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:10:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.40252741200001, 2021-01-15T19:10:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:10:58.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:10:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:10:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:10:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:10:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:10:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:10:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:10:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:10:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:10:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:10:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:10:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:10:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:10:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:10:58.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:10:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:10:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:10:58.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:10:58.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:58.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:10:58.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:10:58.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:10:58.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:10:58.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:10:58.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:10:58.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:10:58.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:10:58.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.766 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.767 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:10:58.767 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:58.767 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 125.439996142, 2021-01-15T19:10:58.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:10:58.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:10:58.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.769 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:58.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:10:58.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:10:58.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:10:58.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:10:58.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:10:58.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:10:58.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:10:58.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:10:58.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:10:58.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:10:58.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:10:58.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:10:58.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:58.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:10:58.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:10:58.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:10:58.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:10:58.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:10:58.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:10:58.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:10:58.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:10:58.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:10:58.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:10:58.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:10:58.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:10:58.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:10:58.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:10:58.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.783 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.784 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:10:58.784 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:58.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 125.439829549, 2021-01-15T19:10:58.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:10:58.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:58.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:10:58.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.786 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:58.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:10:58.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:10:58.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:10:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:10:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:10:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:10:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:10:58.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:10:58.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:10:58.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:10:58.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:10:58.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:10:58.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:58.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:10:58.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:10:58.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:10:58.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:10:58.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:58.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:10:58.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:10:58.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:10:58.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:10:58.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:58.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:10:58.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:10:58.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:10:58.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:10:58.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:10:58.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:10:58.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:10:58.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:10:58.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.802 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:58.802 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:58.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.24129075499999, 2021-01-15T19:10:58.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:10:58.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:10:58.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.804 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:58.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:10:58.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:10:58.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:10:58.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:10:58.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:10:58.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:10:58.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:10:58.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:10:58.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:10:58.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:10:58.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:10:58.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:10:58.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:10:58.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:10:58.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:10:58.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:10:58.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:10:58.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:10:58.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:10:58.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:10:58.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:10:58.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:10:58.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:10:58.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:10:58.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:10:58.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:10:58.819 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.819 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.820 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:58.820 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:58.821 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.23902071000001, 2021-01-15T19:10:58.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:10:58.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.821 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.822 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:10:58.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.822 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:58.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:10:58.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:10:58.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:10:58.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:10:58.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:10:58.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:10:58.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:10:58.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:10:58.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:10:58.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:10:58.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:10:58.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:10:58.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:10:58.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:10:58.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:10:58.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:10:58.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:10:58.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:10:58.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:10:58.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:10:58.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:10:58.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:10:58.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:10:58.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:10:58.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:10:58.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:10:58.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.836 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.837 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:58.837 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:58.838 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.234482519, 2021-01-15T19:10:58.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:10:58.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.838 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.839 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:10:58.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.839 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:58.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:10:58.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:10:58.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:10:58.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:10:58.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:10:58.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:10:58.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:10:58.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:10:58.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:10:58.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:10:58.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:10:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:10:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:10:58.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:10:58.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:10:58.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:10:58.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:10:58.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:10:58.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:10:58.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:10:58.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:10:58.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:10:58.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:10:58.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:10:58.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:10:58.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:10:58.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.854 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:58.855 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:58.855 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.231896647, 2021-01-15T19:10:58.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:10:58.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:10:58.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.857 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:58.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:10:58.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:10:58.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:10:58.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:10:58.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:10:58.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:10:58.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:10:58.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:10:58.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:10:58.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:10:58.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:10:58.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:10:58.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:10:58.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:10:58.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:10:58.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:10:58.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:10:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:10:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:10:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:10:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:10:58.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:10:58.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:10:58.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:10:58.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:10:58.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:10:58.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.872 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:10:58.872 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:58.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.231522312, 2021-01-15T19:10:58.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:10:58.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:10:58.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.874 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:58.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:10:58.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:10:58.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:10:58.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:10:58.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:10:58.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:10:58.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:10:58.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:10:58.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:10:58.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:10:58.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:10:58.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:10:58.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:10:58.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:10:58.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:10:58.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:10:58.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:10:58.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:10:58.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:10:58.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:10:58.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:10:58.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:10:58.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:10:58.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:10:58.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:10:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:10:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.888 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.889 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:10:58.889 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:58.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 106.231385817, 2021-01-15T19:10:58.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:10:58.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:58.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:10:58.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.891 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:58.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:10:58.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:10:58.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.894 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:10:58.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:10:58.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:10:58.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:10:58.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:10:58.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:10:58.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:10:58.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:10:58.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:10:58.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:10:58.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:10:58.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:10:58.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:10:58.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:10:58.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:58.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:10:58.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:10:58.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:10:58.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:10:58.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:10:58.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:10:58.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:10:58.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:10:58.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.905 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.905 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.905 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:10:58.906 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:58.906 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 104.24898809699999, 2021-01-15T19:10:58.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:10:58.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:58.907 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.907 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.907 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:10:58.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.908 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:58.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:10:58.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:10:58.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:10:58.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:10:58.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:10:58.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:10:58.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:10:58.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:10:58.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:10:58.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:10:58.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:10:58.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:10:58.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:58.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:10:58.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:10:58.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:10:58.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:10:58.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:58.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:10:58.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:10:58.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:10:58.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:10:58.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:10:58.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:10:58.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:10:58.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:10:58.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.921 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.922 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:10:58.922 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:58.922 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.102235039, 2021-01-15T19:10:58.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:10:58.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:58.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.923 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.924 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:10:58.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.924 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:58.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:10:58.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:10:58.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:10:58.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:10:58.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:10:58.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:10:58.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:10:58.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:10:58.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:10:58.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:10:58.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:10:58.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:10:58.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:10:58.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:10:58.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:58.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:10:58.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:10:58.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.934 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:58.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:10:58.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:10:58.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.935 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:58.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:10:58.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:10:58.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:10:58.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:10:58.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.937 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.938 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:10:58.938 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 37.102072106999998, 2021-01-15T19:10:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:10:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:58.939 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:58.940 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:58.940 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:58.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:10:58.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.940 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:58.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:10:58.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:10:58.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:10:58.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:10:58.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:10:58.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:10:58.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:10:58.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:10:58.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:10:58.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:10:58.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:10:58.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:10:58.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:58.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:10:58.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:10:58.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:10:58.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:10:58.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.950 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:58.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:10:58.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:10:58.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.951 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:10:58.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:10:58.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:10:58.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:10:58.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:10:58.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:10:58.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:10:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:58.956 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 31.328399470000001, 2021-01-15T19:10:58.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:10:58.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:58.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:58.956 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:58.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:58.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:58.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:58.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:58.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:10:58.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:10:58.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:58.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:10:58.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:10:58.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:58.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:10:58.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:10:58.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:58.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:10:58.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:10:58.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:58.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:10:58.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:10:58.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:10:58.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:10:58.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:10:58.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:10:58.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:10:58.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:58.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:10:58.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:10:58.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:58.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:10:58.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:10:58.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:58.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:58.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:58.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:10:58.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:10:58.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:58.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:58.969 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:58.970 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:10:58.969+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:10:58.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:10:59.071 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:59.071+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:59.144 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:59.145 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:10:59.145 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:59.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:10:59.172 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:10:59.171+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:10:59.273 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:10:59.272+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:10:59.287 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:59.288 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:10:59.288 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:10:59.288 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:10:59.289 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:59.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:10:59.374 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:59.372+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:59.403 INFO:teuthology.orchestra.run.smithi001.stderr:Nothing to do for non-filestore backend 2021-01-15T19:10:59.434 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:10:59.434 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:10:59.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:10:59.434 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:10:59.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:10:59.435 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:59.435 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 140.18267156600001, 2021-01-15T19:10:59.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:10:59.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:10:59.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.437 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:59.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:10:59.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:10:59.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:10:59.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:10:59.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:10:59.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:10:59.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:10:59.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:10:59.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:10:59.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:10:59.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:10:59.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:10:59.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:10:59.446 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:10:59.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:10:59.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:10:59.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:10:59.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:10:59.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:10:59.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:10:59.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:10:59.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:10:59.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.450 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.451 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.451 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:10:59.451 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:59.452 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 140.18248062200001, 2021-01-15T19:10:59.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:10:59.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:59.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.453 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:10:59.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.454 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:59.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:10:59.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:10:59.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:10:59.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:10:59.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:10:59.458 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:10:59.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:10:59.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:10:59.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:10:59.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:10:59.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:10:59.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:10:59.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:59.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:10:59.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:10:59.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:10:59.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:10:59.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:59.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:10:59.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:10:59.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:10:59.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:10:59.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:10:59.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:10:59.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:10:59.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:10:59.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.468 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.468 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.469 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.469 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:59.469 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.24352852499999, 2021-01-15T19:10:59.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:10:59.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.470 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.471 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:10:59.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.471 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:59.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.476 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.476 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:10:59.476 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:10:59.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.477 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.477 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.477 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:10:59.478 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:10:59.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.478 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:10:59.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:10:59.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:10:59.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:10:59.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:10:59.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:10:59.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:10:59.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:10:59.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:10:59.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:10:59.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:10:59.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:10:59.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:10:59.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:10:59.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:10:59.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:10:59.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:10:59.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:10:59.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.490 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.490 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.491 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:59.491 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:59.491 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.24080806399999, 2021-01-15T19:10:59.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:10:59.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:59.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.492 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.493 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:10:59.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.493 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:59.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:10:59.496 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:10:59.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.496 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.497 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:10:59.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:10:59.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.498 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:10:59.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:10:59.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:10:59.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:10:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:10:59.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:10:59.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:10:59.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:10:59.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:59.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:10:59.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:10:59.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:10:59.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:10:59.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:59.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:10:59.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:10:59.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:10:59.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:10:59.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:10:59.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:10:59.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:10:59.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:10:59.509 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.510 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.510 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.511 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.511 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:59.511 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.240695485, 2021-01-15T19:10:59.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:10:59.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.512 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:10:59.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.514 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:59.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:10:59.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:10:59.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.517 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:10:59.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:10:59.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:10:59.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:10:59.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:10:59.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:10:59.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:10:59.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:10:59.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:10:59.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:10:59.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:10:59.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:10:59.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:10:59.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:10:59.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:10:59.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:10:59.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:10:59.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:10:59.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:10:59.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:10:59.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.530 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:59.530 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:59.530 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.240565581, 2021-01-15T19:10:59.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:10:59.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:59.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.532 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:10:59.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.532 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:59.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:10:59.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:10:59.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:10:59.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:10:59.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:10:59.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:10:59.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:10:59.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:10:59.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:10:59.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:10:59.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:10:59.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:10:59.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:10:59.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:10:59.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:10:59.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:10:59.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:10:59.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:59.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:10:59.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:10:59.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:10:59.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:10:59.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:10:59.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:10:59.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:10:59.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:10:59.548 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.549 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.549 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.550 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.550 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:59.550 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.21736031200001, 2021-01-15T19:10:59.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:10:59.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.551 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:10:59.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.552 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:59.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:10:59.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:10:59.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:10:59.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:10:59.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:10:59.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:10:59.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:10:59.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:10:59.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:10:59.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:10:59.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:10:59.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:10:59.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:10:59.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:10:59.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:10:59.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:10:59.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:10:59.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:10:59.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:10:59.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:10:59.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:10:59.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:10:59.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.568 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.568 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:59.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.21724211399999, 2021-01-15T19:10:59.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:10:59.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.571 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:10:59.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.571 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:59.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:10:59.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:10:59.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:10:59.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:10:59.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:10:59.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:10:59.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:10:59.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:10:59.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:10:59.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:10:59.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:10:59.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:10:59.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:10:59.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:10:59.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:10:59.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:10:59.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:10:59.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:10:59.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:10:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:10:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:10:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:10:59.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.21712942400001, 2021-01-15T19:10:59.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:10:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.589 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:10:59.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:10:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:10:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:10:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:10:59.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:10:59.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:10:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:10:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:10:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:10:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:10:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:10:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:10:59.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:10:59.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:10:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:10:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:10:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:10:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:10:59.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:10:59.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:10:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:10:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:10:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:10:59.607 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.211527674, 2021-01-15T19:10:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:10:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:10:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:10:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:10:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:10:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:10:59.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:10:59.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:10:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:10:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:10:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:10:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:10:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:10:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:10:59.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:10:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:10:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.620 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.20305895000001, 2021-01-15T19:10:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:10:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:10:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:10:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:10:59.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:10:59.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:10:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:10:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:10:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:10:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:10:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:10:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:10:59.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:10:59.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:10:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:10:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:10:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:10:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:10:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:10:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:10:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:10:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:10:59.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:10:59.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:10:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:10:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.19854036699999, 2021-01-15T19:10:59.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:10:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.640 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:10:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:10:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:10:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:10:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:10:59.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:10:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:10:59.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:10:59.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:10:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:10:59.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:10:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:10:59.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:10:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:10:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:10:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:10:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:10:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:10:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:10:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:10:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:10:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:10:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:10:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.654 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.655 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:10:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 135.19352259799999, 2021-01-15T19:10:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:10:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:10:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:10:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:10:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:10:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:10:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:10:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:10:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:10:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:10:59.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:10:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:10:59.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:10:59.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:10:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:10:59.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:10:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:10:59.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:10:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:10:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:10:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:10:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:10:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:10:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:59.671 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.22639852, 2021-01-15T19:10:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:10:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.672 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:10:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.673 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:59.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:10:59.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:10:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:10:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:10:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:10:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:10:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:10:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:10:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:10:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:10:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:10:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:10:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:10:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:10:59.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:10:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:10:59.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:10:59.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:10:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:10:59.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:10:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:10:59.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:10:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:10:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:10:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:10:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:10:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.689 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:10:59.690 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 111.249211872, 2021-01-15T19:10:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:10:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.691 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:10:59.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:59.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:10:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:10:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:10:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:10:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:10:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:10:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:10:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:10:59.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:10:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:10:59.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:10:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:10:59.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:10:59.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:10:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:10:59.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:10:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:10:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:10:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:10:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:10:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:10:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:10:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:10:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:10:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:10:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:10:59.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.709 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:10:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.218664851, 2021-01-15T19:10:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:10:59.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:10:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:10:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:10:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:10:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:10:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:10:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:10:59.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:10:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:10:59.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:10:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:10:59.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:10:59.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:10:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:10:59.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:10:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:10:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:10:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:10:59.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:10:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:10:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:10:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:10:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:10:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:10:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:10:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:10:59.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.228006876000002, 2021-01-15T19:10:59.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:10:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:10:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:10:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:10:59.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:10:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:10:59.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:10:59.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:10:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:10:59.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:10:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:10:59.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:10:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:10:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:10:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:10:59.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:10:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:10:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:10:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:10:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:10:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:10:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:10:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:10:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:10:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:10:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:10:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:10:59.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:10:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:10:59.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.227867343, 2021-01-15T19:10:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:10:59.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:10:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:10:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:10:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:10:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:10:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:10:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:10:59.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:10:59.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:10:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:10:59.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:10:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:10:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:10:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:59.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:10:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:10:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:10:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:10:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:10:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:10:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:10:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:10:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:10:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:10:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:10:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:10:59.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:10:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:10:59.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:10:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 56.227752029999998, 2021-01-15T19:10:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:10:59.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:10:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:10:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:10:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:10:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:10:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:10:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:10:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:10:59.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:10:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:10:59.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:10:59.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:10:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:10:59.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:10:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:10:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:10:59.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:10:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:10:59.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:10:59.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:10:59.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:10:59.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:10:59.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:10:59.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:10:59.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:10:59.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:10:59.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:10:59.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:10:59.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:10:59.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:10:59.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:10:59.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:10:59.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.783 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.784 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:10:59.785 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:59.785 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 20.114280734000001, 2021-01-15T19:10:59.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:10:59.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:10:59.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:10:59.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:10:59.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:10:59.786 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:10:59.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:10:59.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.787 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:10:59.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:10:59.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:59.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:10:59.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:10:59.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:10:59.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:10:59.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:10:59.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:10:59.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:10:59.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:10:59.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:10:59.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:10:59.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:10:59.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:10:59.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:10:59.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:10:59.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:10:59.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:10:59.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:10:59.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:10:59.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:10:59.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:10:59.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:10:59.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:10:59.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:10:59.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:10:59.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:10:59.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:10:59.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:10:59.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:10:59.798 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:10:59.798 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:10:59.800 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:10:59.798+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:10:59.800 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:10:59.901 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:10:59.900+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:10:59.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:00.002 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:00.001+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:00.075 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:00.075 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:00.076 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:00.076 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:00.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:00.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:00.126 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:00.125+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:00.203 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:00.202+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:00.222 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:11:00.251 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:00.255 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:00.255 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:00.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:00.256 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:00.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.256 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:00.256 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:00.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.06900790500001, 2021-01-15T19:11:00.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:00.257 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.257 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:00.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:00.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:00.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:00.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:00.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:00.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:00.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:00.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:00.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:00.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:00.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:00.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:00.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:00.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:00.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:00.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:00.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:00.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.269 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.270 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:00.270 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:00.270 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 112.06884496399999, 2021-01-15T19:11:00.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:00.270 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.271 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.271 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.272 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.272 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.272 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:00.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.272 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:00.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:00.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:00.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:00.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:00.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:00.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:00.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:00.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:00.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:00.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:00.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:00.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:00.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:00.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:00.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:00.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:00.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.282 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.283 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:00.283 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:00.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 107.90780231799999, 2021-01-15T19:11:00.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:00.284 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.284 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.285 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.285 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:00.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:00.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:00.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:00.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:00.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:00.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:00.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:00.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:00.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:00.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:00.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:00.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:00.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:00.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:00.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:00.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:00.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:00.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:00.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:00.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:00.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:00.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:00.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:00.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:00.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:00.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:00.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:00.301 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.302 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:00.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:00.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 107.907694862, 2021-01-15T19:11:00.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:00.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.305 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.306 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:00.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:00.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:00.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:00.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:00.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:00.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:00.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:00.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:00.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:00.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:00.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:00.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:00.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:00.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:00.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:00.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:00.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:00.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:00.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:00.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:00.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:00.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:00.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:00.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:00.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:00.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:00.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:00.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:00.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:00.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:00.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:00.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.326 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:00.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:00.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 107.77462799600001, 2021-01-15T19:11:00.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:00.326 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.327 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.327 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.327 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:00.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.328 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:00.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:00.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:00.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:00.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:00.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:00.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:00.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:00.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:00.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:00.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:00.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:00.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:00.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:00.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:00.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:00.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:00.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:00.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:00.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:00.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:00.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:00.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:00.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:00.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:00.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:00.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:00.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.344 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.344 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.345 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:00.345 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:00.345 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.878660313, 2021-01-15T19:11:00.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:00.345 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.346 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:00.346 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.346 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.346 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.346 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:00.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.347 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:00.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:00.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:00.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:00.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:00.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:00.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:00.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:00.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:00.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:00.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:00.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:00.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:00.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:00.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:00.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:00.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:00.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:00.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:00.358 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.358 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.358 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.359 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:00.359 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:00.360 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.878530183, 2021-01-15T19:11:00.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:00.360 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.360 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:00.360 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.361 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.361 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.361 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:00.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.361 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:00.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:00.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:00.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:00.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:00.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:00.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:00.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:00.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:00.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:00.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:00.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:00.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:00.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:00.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:00.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:00.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:00.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:00.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:00.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:00.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:00.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 102.878444774, 2021-01-15T19:11:00.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:00.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:00.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:00.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.375 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:00.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:00.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:00.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:00.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:00.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:00.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:00.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:00.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:00.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:00.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:00.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:00.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:00.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:00.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:00.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:00.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:00.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 97.047270385999994, 2021-01-15T19:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 97.047181737000002, 2021-01-15T19:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.063673679000004, 2021-01-15T19:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:00.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:00.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.443 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.443 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.444 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:00.444 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:00.444 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.063541603999994, 2021-01-15T19:11:00.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.060406087000004, 2021-01-15T19:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.060302061000002, 2021-01-15T19:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.049315311999997, 2021-01-15T19:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.049219790999999, 2021-01-15T19:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:00.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:00.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:00.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:00.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:00.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:00.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.529 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.530 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:00.530 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:00.530 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.049108644999997, 2021-01-15T19:11:00.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:00.531 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.531 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.532 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:00.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.532 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:00.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:00.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:00.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:00.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:00.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:00.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:00.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:00.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:00.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:00.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:00.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:00.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:00.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:00.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:00.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:00.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:00.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:00.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:00.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:00.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:00.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:00.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.545 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.545 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.545 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:00.546 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:00.546 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.957334322999998, 2021-01-15T19:11:00.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:00.546 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.546 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.547 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.547 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.547 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.547 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:00.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.548 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:00.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:00.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:00.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:00.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:00.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:00.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:00.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:00.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:00.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:00.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:00.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:00.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:00.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:00.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:00.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:00.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:00.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:00.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:00.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:00.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:00.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:00.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:00.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.560 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.561 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:00.561 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:00.562 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.956051184000003, 2021-01-15T19:11:00.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:00.562 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.562 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:00.562 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.563 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.563 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:00.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.563 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:00.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:00.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:00.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:00.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:00.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:00.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:00.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:00.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:00.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:00.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:00.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:00.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:00.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:00.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:00.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:00.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:00.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:00.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:00.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:00.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:00.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:00.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:00.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:00.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:00.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:00.575 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.576 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.576 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:00.577 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:00.577 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.050923215999999, 2021-01-15T19:11:00.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:00.577 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:00.577 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:00.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:00.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:00.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:00.578 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:00.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.579 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:00.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:00.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:00.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:00.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:00.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:00.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:00.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:00.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:00.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:00.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:00.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:00.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:00.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:00.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:00.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:00.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:00.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:00.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:00.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:00.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:00.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:00.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:00.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:00.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:00.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:00.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:00.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:00.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:00.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:00.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:00.589 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:00.589 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:00.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:00.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:00.591 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:00.592 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:00.592 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:00.594 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:00.594 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:00.595 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:00.589+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:00.596 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:00.691 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:00.690+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:00.722 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:00.723 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:00.732 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:00.792 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:00.790+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:00.893 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:00.892+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:00.896 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:00.896 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:00.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:00.994 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:00.993+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:01.071 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:01.071 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:01.081 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:01.122 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:01.121+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:01.196 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:01.195+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:01.246 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:01.247 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:01.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:01.296 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:01.295+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:01.323 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:11:01.397 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:01.396+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:01.421 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:01.422 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:01.430 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:01.514 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:01.513+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:01.596 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:01.596 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:01.598 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:01.597+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:01.605 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:01.699 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:01.698+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:01.769 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:01.770 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:01.779 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:01.823 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:01.822+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:01.900 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:01.899+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:01.904 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:01.904 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:01.914 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:02.001 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:02.000+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:02.078 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:02.079 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:02.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:02.129 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:02.128+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:02.203 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:02.202+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:02.233 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:02.234 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:02.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:02.327 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:02.326+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:02.405 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:02.402+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:02.418 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:02.418 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:02.428 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:02.506 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:02.504+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:02.595 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:02.599 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:02.599 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:02.607 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:02.605+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:02.610 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:02.708 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:02.706+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:02.768 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:02.790 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:02.803 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:02.804 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:02.804 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:02.808 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:02.807+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:02.811 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:02.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:02.833 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:02.853 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:02.874 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:02.878 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:02.878 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:02.910 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:02.908+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:02.965 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:02.966 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:02.966 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:02.976 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:03.010 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:03.009+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:03.109 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:03.109 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.110 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 143.857725053, 2021-01-15T19:11:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:11:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:11:03.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:03.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:11:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:11:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:11:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:11:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:11:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:11:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:11:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:11:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:11:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:11:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:11:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:11:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:11:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:11:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:11:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:11:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:11:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:11:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:11:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:11:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:11:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.125 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:03.126 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 143.857534109, 2021-01-15T19:11:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:11:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:11:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:11:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:11:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:11:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:11:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:11:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:11:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:11:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:11:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:11:03.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:11:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:11:03.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:11:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:11:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:11:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:11:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:11:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:11:03.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:11:03.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:11:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:11:03.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:11:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:11:03.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:11:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.918582012, 2021-01-15T19:11:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:11:03.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:11:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:11:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:11:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:11:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:11:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:11:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:11:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:11:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:11:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:11:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:11:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:11:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:11:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:11:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:11:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:11:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:11:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:11:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:11:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:11:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:11:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:11:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.157 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:03.158 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.91586155100001, 2021-01-15T19:11:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:11:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.159 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:11:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:11:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:11:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:11:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:11:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:11:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:11:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:11:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:11:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:11:03.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:11:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:11:03.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:11:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:11:03.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:11:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:11:03.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:11:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:11:03.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:11:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:11:03.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:11:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:11:03.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:11:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:11:03.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.175 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.91574897199999, 2021-01-15T19:11:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:11:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:11:03.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:11:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:11:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:11:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:11:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:11:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:11:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:11:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:11:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:11:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:11:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:11:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:11:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:11:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:11:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:11:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:11:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:11:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:11:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:11:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:11:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:11:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.190 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:03.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.91561906800001, 2021-01-15T19:11:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:11:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:11:03.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:11:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:11:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:11:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:11:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:11:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:11:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:11:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:11:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:11:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:11:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:11:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:03.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:11:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:11:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:11:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:11:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:11:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:11:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:11:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:11:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:11:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:11:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.206 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:11:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:11:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:03.208 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.892413799, 2021-01-15T19:11:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:11:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.209 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:11:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:11:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:11:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:11:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:11:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:11:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:11:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:11:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:11:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:11:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:11:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:11:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:11:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:11:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:11:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:11:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:11:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:11:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:11:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:11:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:11:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.223 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.892295601, 2021-01-15T19:11:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:11:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.224 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:11:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.225 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:03.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:11:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:11:03.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:11:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:11:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:11:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:11:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:11:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:11:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:11:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:11:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:11:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:11:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:11:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:11:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:11:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:11:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:11:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:11:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:11:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:11:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:11:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.89218291099999, 2021-01-15T19:11:03.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:11:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.240 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:11:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.241 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.242 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:11:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.243 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:11:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:11:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.244 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:11:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:11:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:11:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:11:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:11:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:11:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:11:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:11:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:11:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:11:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:11:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:11:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:11:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:11:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:11:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:11:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:11:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:11:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.253 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:11:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:03.254 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.88658116100001, 2021-01-15T19:11:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:11:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:11:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:11:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:11:03.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:11:03.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:11:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:11:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:11:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:11:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:11:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:11:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:11:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:11:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:11:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:11:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.263 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.264 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.878112437, 2021-01-15T19:11:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:11:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.265 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:11:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.266 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:03.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:11:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:11:03.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:11:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:11:03.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:11:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:11:03.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:11:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:11:03.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:11:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:11:03.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:11:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:11:03.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:11:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:11:03.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:11:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:11:03.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:11:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:11:03.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:11:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:11:03.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:11:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.87359385400001, 2021-01-15T19:11:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:11:03.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:11:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:11:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:11:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:11:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:11:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:11:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:11:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:11:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:11:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:11:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:11:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:11:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:11:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:11:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:11:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:11:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:11:03.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:11:03.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:11:03.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:11:03.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:11:03.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:11:03.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.294 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.295 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:03.295 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:03.295 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 138.868576085, 2021-01-15T19:11:03.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:11:03.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:03.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.297 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:11:03.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.297 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:03.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:11:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:11:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:11:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:11:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:11:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:11:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:11:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:11:03.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:11:03.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:11:03.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:11:03.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:11:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:11:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:11:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:11:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:11:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:11:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:11:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:11:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.308 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:11:03.309 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:03.310 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 124.901452007, 2021-01-15T19:11:03.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:11:03.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:11:03.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:03.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:03.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:11:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:11:03.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:11:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:11:03.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:11:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:11:03.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:11:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:11:03.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:11:03.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:11:03.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:11:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:11:03.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:11:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:11:03.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:11:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:11:03.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:11:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:11:03.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:11:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:11:03.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:11:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:11:03.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:11:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:11:03.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:11:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:11:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 114.924265359, 2021-01-15T19:11:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:11:03.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:11:03.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.329 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.329 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:03.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:11:03.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:11:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:11:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:11:03.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:11:03.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:11:03.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:11:03.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:11:03.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:11:03.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:11:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:11:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:03.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:11:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:11:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:11:03.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:11:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:11:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:11:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:11:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:11:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:11:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:11:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:11:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:11:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:11:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:11:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:11:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.893718337999999, 2021-01-15T19:11:03.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:11:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.345 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:11:03.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.346 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:11:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:11:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:11:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:11:03.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:11:03.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:11:03.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:11:03.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:11:03.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:11:03.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:11:03.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:11:03.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:11:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:11:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:11:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:11:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:11:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:11:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:11:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:11:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:11:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:11:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:11:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:11:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.359 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:03.360 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.903060363000002, 2021-01-15T19:11:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:11:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.361 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:11:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.362 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:03.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:11:03.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:03.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:11:03.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:11:03.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:11:03.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:11:03.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:11:03.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:11:03.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:11:03.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:11:03.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:11:03.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:03.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:11:03.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:11:03.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.371 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.371 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:11:03.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:11:03.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:11:03.373 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:11:03.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:11:03.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:11:03.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:11:03.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:11:03.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:11:03.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:11:03.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.376 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:11:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:11:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.378 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:03.378 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:03.378 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.902920829999999, 2021-01-15T19:11:03.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:11:03.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.379 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.380 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:11:03.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.380 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:03.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.382 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:03.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:11:03.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:11:03.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:11:03.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:11:03.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:11:03.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.385 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:11:03.385 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:11:03.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.386 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.386 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:11:03.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:11:03.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.386 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.387 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.387 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:11:03.387 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:11:03.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:03.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:11:03.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:11:03.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:11:03.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:11:03.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:11:03.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:11:03.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:11:03.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:11:03.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:11:03.392 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:11:03.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:11:03.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:11:03.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:11:03.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:11:03.394 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.395 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 59.902805516999997, 2021-01-15T19:11:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:11:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:11:03.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.398 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:03.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:03.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:11:03.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.400 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.400 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.400 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:11:03.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:11:03.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.401 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.401 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:11:03.401 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:11:03.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.402 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.402 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:11:03.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:11:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:03.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:11:03.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:11:03.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:11:03.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:03.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:11:03.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:11:03.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.406 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:11:03.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:11:03.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:11:03.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:11:03.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:11:03.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:11:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:11:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:11:03.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:03.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:11:03.410 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:11:03.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.411 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.411 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:11:03.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:11:03.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.412 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.412 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:11:03.413 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:03.413 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 23.789334221000001, 2021-01-15T19:11:03.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:11:03.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:03.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:03.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:03.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:03.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:11:03.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:03.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:03.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:11:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:11:03.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:11:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:11:03.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:11:03.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:03.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:11:03.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:11:03.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:11:03.420 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:11:03.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:03.421 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:11:03.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:11:03.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:03.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:11:03.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:11:03.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:03.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:03.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:03.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:11:03.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:11:03.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.424 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:03.424 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:03.425 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:03.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:11:03.427 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:03.425+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:03.427 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:03.527 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:03.525+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:03.572 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:03.572 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:03.572 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:03.572 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:03.581 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:03.627 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:03.626+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:03.728 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:03.726+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:03.745 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:03.745 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:03.745 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:03.746 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:03.746 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:03.754 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:03.839 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:03.837+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:03.922 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:03.923 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:03.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:03.923 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:03.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.924 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:03.924 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:03.924 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.735888778, 2021-01-15T19:11:03.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:03.925 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:03.925 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:03.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:03.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:03.925 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.926 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:03.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.926 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:03.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:03.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:03.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:03.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:03.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:03.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:03.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:03.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:03.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:03.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:03.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:03.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:03.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:03.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:03.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:03.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:03.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:03.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:03.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:03.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:03.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:03.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:03.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:03.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:03.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:03.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.936 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:03.936 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.936 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:03.937 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:03.937 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.735725837, 2021-01-15T19:11:03.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:03.937 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:03.937 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:03.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:03.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:03.938 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.938 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:03.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:03.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:03.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:03.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:03.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:03.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:03.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:03.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:03.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:03.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:03.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:03.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:03.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:03.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:03.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:03.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:03.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:03.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:03.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:03.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:03.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:03.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:03.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:03.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.57468319100001, 2021-01-15T19:11:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:03.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:03.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:03.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:03.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:03.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:03.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:03.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:03.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:03.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:03.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:03.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:03.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:03.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:03.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:03.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:03.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:03.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:03.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:03.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:03.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:03.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:03.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:03.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:03.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:03.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:03.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:03.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:03.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:03.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:03.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:03.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:03.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:03.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:03.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:03.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:03.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:03.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:03.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.967 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:03.967 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:03.968 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.574575735, 2021-01-15T19:11:03.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:03.968 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:03.968 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:03.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:03.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:03.969 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.969 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:03.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.970 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:03.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:03.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:03.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:03.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:03.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:03.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:03.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:03.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:03.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:03.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:03.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:03.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:03.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:03.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:03.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:03.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:03.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:03.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:03.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:03.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:03.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:03.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:03.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:03.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:03.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:03.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:03.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:03.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:03.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:03.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:03.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:03.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:03.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:03.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:03.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:03.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:03.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:03.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:03.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:03.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:03.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.985 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:03.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:03.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.986 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:03.986 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:03.986 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.441508869, 2021-01-15T19:11:03.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:03.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:03.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:03.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:03.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:03.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:03.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:03.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:03.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:03.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:03.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:03.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:03.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:03.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:03.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:03.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:03.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:03.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:03.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:03.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:03.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:03.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:03.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:03.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:03.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:03.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:03.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:03.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:03.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:03.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:03.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:03.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:03.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:03.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:03.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:03.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:03.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:03.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:03.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:03.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:03.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:04.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:04.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:04.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:04.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:04.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:04.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:04.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:04.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:04.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:04.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.54554118599999, 2021-01-15T19:11:04.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:04.005 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.005 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:04.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.006 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:04.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:04.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:04.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:04.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:04.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:04.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:04.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:04.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:04.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:04.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:04.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:04.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:04.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:04.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:04.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:04.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:04.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:04.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:04.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:04.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:04.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.017 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.017 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:04.018 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:04.018 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.54541105600001, 2021-01-15T19:11:04.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:04.018 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.018 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:04.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.019 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:04.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.020 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:04.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:04.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:04.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:04.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:04.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:04.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:04.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:04.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:04.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:04.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:04.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:04.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:04.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:04.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:04.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:04.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:04.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:04.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:04.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:04.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.030 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 106.545325647, 2021-01-15T19:11:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:04.032 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.032 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:04.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.032 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.033 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:04.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.033 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:04.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:04.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:04.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:04.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:04.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:04.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:04.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:04.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:04.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:04.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:04.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:04.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:04.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:04.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:04.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:04.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:04.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:04.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:04.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:04.043 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.044 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.044 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:04.045 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:04.045 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.714151259, 2021-01-15T19:11:04.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:04.045 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.045 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.046 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.046 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:04.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.047 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:04.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:04.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:04.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:04.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:04.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:04.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:04.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:04.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:04.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:04.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:04.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:04.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:04.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:04.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:04.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:04.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:04.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:04.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:04.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:04.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:04.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:04.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:04.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:04.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:04.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:04.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:04.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:04.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:04.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:04.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:04.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:04.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.064 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:04.064 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:04.065 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.71406261, 2021-01-15T19:11:04.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:04.065 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.065 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.066 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:04.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:04.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:04.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:04.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:04.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:04.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:04.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:04.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:04.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:04.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:04.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:04.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:04.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:04.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:04.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:04.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:04.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:04.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:04.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:04.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:04.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:04.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:04.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:04.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:04.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:04.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:04.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:04.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:04.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:04.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:04.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:04.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:04.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.083 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.083 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.084 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:04.084 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:04.084 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.730554552000001, 2021-01-15T19:11:04.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:04.084 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.085 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.085 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.085 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.085 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.085 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:04.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.086 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:04.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:04.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:04.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:04.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:04.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:04.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:04.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:04.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:04.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:04.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:04.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:04.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:04.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:04.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:04.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:04.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:04.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:04.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:04.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:04.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:04.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:04.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:04.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:04.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:04.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.101 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.101 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:04.102 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:04.102 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.730422477000005, 2021-01-15T19:11:04.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:04.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.103 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.103 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.103 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.103 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:04.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.104 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:04.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:04.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:04.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:04.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:04.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:04.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:04.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:04.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:04.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:04.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:04.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:04.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:04.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:04.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:04.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:04.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:04.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:04.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:04.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:04.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:04.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:04.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:04.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:04.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:04.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:04.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:04.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:04.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:04.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:04.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:04.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.119 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.120 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:04.120 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:04.120 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.727286960000001, 2021-01-15T19:11:04.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:04.121 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.121 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.121 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.122 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:04.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:04.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:04.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:04.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:04.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:04.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:04.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:04.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:04.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:04.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:04.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:04.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:04.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:04.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:04.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:04.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:04.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:04.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:04.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:04.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:04.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:04.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:04.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:04.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:04.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:04.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.727182933999998, 2021-01-15T19:11:04.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:04.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.139 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:04.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:04.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:04.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:04.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:04.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:04.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:04.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:04.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:04.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:04.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:04.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:04.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:04.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:04.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:04.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:04.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:04.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:04.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:04.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.716196185000001, 2021-01-15T19:11:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:04.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:04.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:04.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:04.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:04.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:04.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:04.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:04.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:04.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:04.173 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.716100664000002, 2021-01-15T19:11:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.174 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:04.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:04.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:04.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:04.188 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.715989518000001, 2021-01-15T19:11:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.189 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:04.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:04.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:04.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:04.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:04.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.202 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.624215196000002, 2021-01-15T19:11:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.203 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.204 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.217 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:04.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.622932057, 2021-01-15T19:11:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:04.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:04.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:04.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:04.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:04.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:04.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:04.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:04.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:04.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:04.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:04.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.717804088999999, 2021-01-15T19:11:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:04.234 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.235 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.246 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.246 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:04.247 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:04.248 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:04.245+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:04.336 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:11:04.348 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:04.346+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:04.415 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:04.415 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:04.415 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:04.415 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:04.424 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:04.466 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:04.463+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:04.550 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:04.548+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:04.591 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:04.592 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:04.599 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:04.684 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:04.682+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:04.752 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:04.751+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:11:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.478838556, 2021-01-15T19:11:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:11:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:11:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:11:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:11:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:11:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:11:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:11:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:11:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:11:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:11:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:11:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:11:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:11:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:11:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:11:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:11:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:11:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:11:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:11:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:11:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:11:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:11:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:11:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:11:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:11:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:11:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:11:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.54707579000001, 2021-01-15T19:11:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:11:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:11:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:11:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:11:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:11:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:11:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:11:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:11:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:11:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:11:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:11:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:11:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:11:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:11:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:11:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:11:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:11:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:11:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:11:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:11:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:11:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:11:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:11:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:11:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:11:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:11:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:11:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.54478774899999, 2021-01-15T19:11:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:11:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:11:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:11:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:11:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:11:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:11:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:11:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:11:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:11:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:11:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:11:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:11:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:11:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:11:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:11:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:11:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:11:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:11:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:11:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:11:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:11:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:11:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:11:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.54219812400001, 2021-01-15T19:11:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:11:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:11:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:11:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:11:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:11:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:11:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:11:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:11:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:11:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:11:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:11:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:11:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:11:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:11:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:11:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:11:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:11:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:11:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:11:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:11:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:11:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:11:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:11:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:11:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.54128984600001, 2021-01-15T19:11:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:11:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:11:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:11:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:11:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:11:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:11:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:11:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:11:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:11:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:11:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:11:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:11:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:11:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:11:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:11:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:11:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:11:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:11:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:11:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:11:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:11:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 151.572605939, 2021-01-15T19:11:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:11:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:11:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:11:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:11:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:11:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:11:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:11:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:11:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:11:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:11:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:11:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:11:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:11:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:11:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:11:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:11:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:11:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:11:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:11:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:11:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:11:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:11:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:11:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:11:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:11:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:11:04.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:11:04.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.868 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 151.57245620200001, 2021-01-15T19:11:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:11:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:11:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:11:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:11:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:11:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:11:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:11:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:11:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:11:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:11:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:11:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:11:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:11:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:11:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:11:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:11:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:11:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:11:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:11:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:11:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:11:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:11:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:11:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:11:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:11:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:11:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:11:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 151.572304304, 2021-01-15T19:11:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:11:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:11:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:11:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:11:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:11:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:11:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:11:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:11:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:11:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:11:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:11:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:11:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:11:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:11:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:11:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:11:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:11:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:11:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:11:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:11:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:11:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:11:04.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:11:04.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:11:04.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:11:04.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:11:04.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:11:04.904 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.905 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 151.572178992, 2021-01-15T19:11:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:11:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.907 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.907 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.908 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:11:04.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.908 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:04.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:04.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:11:04.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:11:04.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:11:04.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:11:04.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:11:04.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:11:04.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:11:04.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:11:04.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:11:04.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:11:04.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:11:04.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:04.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:11:04.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:11:04.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:11:04.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:11:04.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:04.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:11:04.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:11:04.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:11:04.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:11:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:11:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:11:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:11:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:11:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:11:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:11:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:11:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:11:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:11:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.55134641699999, 2021-01-15T19:11:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:11:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:11:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:11:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:11:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:11:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:11:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:11:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:11:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:11:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:11:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:11:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:11:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:11:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:11:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:11:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:11:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:11:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:11:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:11:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:11:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:11:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:11:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:11:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:11:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.54505308500001, 2021-01-15T19:11:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:11:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:11:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:11:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:11:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:11:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:11:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:11:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:11:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:11:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:11:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:11:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:11:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:11:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:11:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:11:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:11:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:11:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:11:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:11:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:11:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:11:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:11:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.58299357199999, 2021-01-15T19:11:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:11:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:11:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:11:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:11:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:11:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:11:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:11:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:11:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:11:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:11:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:11:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:11:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:11:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:11:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:11:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:11:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:11:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:11:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:11:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:11:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:11:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:11:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:11:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:11:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:11:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:11:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:11:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.520621592, 2021-01-15T19:11:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:11:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:11:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:11:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:11:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:11:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:11:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:11:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:11:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:11:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:11:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:11:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:11:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:11:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:11:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:11:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:11:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:11:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:11:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:11:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:11:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:11:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:11:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:11:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.51731909099999, 2021-01-15T19:11:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:11:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:11:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:11:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:11:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:11:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:11:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:11:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:11:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:11:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:11:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:11:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:11:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:11:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:11:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:11:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:11:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:11:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:11:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:11:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:11:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:11:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.51599722500001, 2021-01-15T19:11:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:11:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:11:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:11:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:11:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:11:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:11:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:11:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:11:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:11:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:11:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:11:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:11:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:11:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:11:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:11:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:11:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:11:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:11:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:11:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:11:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:11:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:11:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.51562697200001, 2021-01-15T19:11:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:11:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:11:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:11:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:11:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:11:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:11:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:11:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:11:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:11:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:11:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:11:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:11:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:11:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:11:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:11:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:11:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:11:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:11:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:11:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:11:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:11:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:11:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:11:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.515292579, 2021-01-15T19:11:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:11:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:11:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:11:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:11:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:11:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:11:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:11:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:11:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:11:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:11:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:11:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:11:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:11:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:11:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:11:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:11:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:11:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:11:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:11:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:11:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:11:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:11:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 131.57403131999999, 2021-01-15T19:11:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:11:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:11:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:11:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:11:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:11:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:11:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:11:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:11:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:11:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:11:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:11:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:11:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:11:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:11:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:11:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:11:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:11:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:11:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:11:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:11:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:11:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.581668448, 2021-01-15T19:11:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:11:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:11:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:11:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:11:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:11:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:11:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:11:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:11:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:11:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:11:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:11:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:11:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:11:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:11:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:11:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:11:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:11:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:05.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:11:05.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:11:05.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:11:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:11:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:11:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:11:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:11:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:11:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:11:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:11:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:11:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.551674244, 2021-01-15T19:11:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:11:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:11:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:11:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:11:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:11:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:11:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:11:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:11:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:11:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:11:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:11:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:11:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:11:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:11:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:11:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:11:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:11:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:11:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:11:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:11:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:11:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:05.096 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:05.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:05.098 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:05.098 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:05.098 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.052222 2021-01-15T19:11:05.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.05222188121685334 2021-01-15T19:11:05.100 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:05.101 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:05.096+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:05.199 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:05.198+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:05.267 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:05.267 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:05.268 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:05.277 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:05.287 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:05.308 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:05.321 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:05.319+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:05.328 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:05.350 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:05.370 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:05.371 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:05.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:05.373 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:05.373 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:05.373 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:05.374 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:05.374 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:05.404 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:05.405 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:05.414 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:05.415 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:05.414+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:05.503 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:05.501+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:05.582 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:05.582 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:05.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:05.630 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:05.628+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:05.705 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:05.703+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:05.754 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:05.754 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:05.764 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:05.807 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:05.805+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:05.899 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:05.940 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:05.938+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:06.007 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:06.005+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:06.063 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:06.064 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:06.073 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:06.114 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:06.112+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:06.208 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:06.207+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:06.220 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:06.221 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:06.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:06.309 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:06.308+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:06.338 INFO:teuthology.orchestra.run.smithi001.stderr:set osd.1 primary-affinity to 0.0522219 (834222) 2021-01-15T19:11:06.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:11:06.407 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:06.407 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:06.410 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:06.408+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:06.418 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:06.510 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:06.508+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:06.592 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:06.592 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:06.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:06.646 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:06.645+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:06.712 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:06.711+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:06.736 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:06.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:06.831 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:06.830+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:06.914 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:06.912+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:06.920 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:06.920 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:06.930 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:07.015 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:07.014+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:07.102 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:07.102 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:07.112 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:07.116 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:07.115+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:07.217 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:07.215+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:07.247 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:07.247 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:07.257 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:07.342 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:07.340+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:07.343 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:11:07.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:07.419 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:07.417+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:07.423 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:07.423 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:07.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:07.520 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:07.518+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:07.556 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:07.576 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:07.596 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:07.601 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:07.601 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:07.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:07.602 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:07.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.602 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:07.603 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:07.603 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.415358357, 2021-01-15T19:11:07.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:07.603 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.603 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.604 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:07.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.605 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:07.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:07.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:07.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:07.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:07.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:07.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:07.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:07.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:07.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:07.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:07.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:07.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:07.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:07.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:07.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:07.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:07.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:07.616 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:07.616 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.415195416, 2021-01-15T19:11:07.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:07.617 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.618 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:07.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.619 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:07.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:07.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:07.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:07.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:07.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:07.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:07.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:07.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:07.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:07.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:07.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:07.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:07.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:07.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:07.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:07.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:07.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.630 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.25415277, 2021-01-15T19:11:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:07.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.636 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:07.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:07.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:07.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:07.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:07.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:07.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:07.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:07.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:07.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:07.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:07.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:07.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:07.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:07.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:07.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:07.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:07.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:07.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:07.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:07.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:07.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:07.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:07.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:07.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:07.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.662 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.664 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:07.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:07.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.254045314, 2021-01-15T19:11:07.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:07.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.666 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.667 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.667 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:07.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.668 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:07.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:07.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:07.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:07.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:07.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:07.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:07.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:07.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:07.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:07.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:07.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:07.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:07.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:07.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:07.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:07.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:07.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:07.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:07.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:07.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:07.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:07.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:07.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:07.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:07.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:07.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:07.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:07.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:07.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.687 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:07.687 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:07.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.120978448, 2021-01-15T19:11:07.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:07.688 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.688 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:07.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:07.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:07.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:07.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:07.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:07.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:07.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:07.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:07.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:07.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:07.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 110.22501076499999, 2021-01-15T19:11:07.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:07.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:07.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:07.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:07.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:07.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:07.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:07.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.722 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.723 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:07.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:07.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 110.22488063500001, 2021-01-15T19:11:07.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:07.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:07.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.724 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:07.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.725 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:07.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:07.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:07.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:07.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:07.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:07.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:07.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:07.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:07.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:07.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:07.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:07.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:07.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:07.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:07.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:07.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:07.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:07.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:07.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:07.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:07.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 110.224795226, 2021-01-15T19:11:07.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:07.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:07.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:07.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:07.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:07.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:07.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:07.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:07.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:07.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:07.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:07.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:07.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:07.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:07.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:07.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:07.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:07.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:07.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:07.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:07.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:07.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:07.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:07.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:07.751 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 104.393620838, 2021-01-15T19:11:07.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:07.751 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.751 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.752 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:07.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:07.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:07.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:07.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:07.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:07.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:07.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:07.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:07.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:07.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:07.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:07.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:07.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:07.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:07.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:07.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:07.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:07.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:07.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:07.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:07.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:07.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:07.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:07.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:07.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:07.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:07.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:07.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:07.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:07.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:07.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:07.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:07.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.773 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.775 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:07.775 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:07.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 104.393532189, 2021-01-15T19:11:07.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:07.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.777 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.778 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.779 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:07.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.779 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:07.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:07.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:07.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:07.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:07.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:07.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:07.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:07.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:07.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:07.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:07.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:07.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:07.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:07.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:07.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:07.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:07.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:07.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:07.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:07.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:07.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:07.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:07.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:07.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:07.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:07.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:07.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:07.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:07.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:07.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:07.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:07.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:07.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.812 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.813 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:07.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:07.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 94.410024131, 2021-01-15T19:11:07.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:07.814 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.815 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:07.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.816 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:07.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:07.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:07.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:07.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:07.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:07.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:07.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:07.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:07.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:07.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:07.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:07.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:07.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:07.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:07.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:07.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:07.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:07.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:07.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:07.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:07.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:07.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:07.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:07.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:07.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:07.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:07.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:07.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.831 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:07.831 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:07.831 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 94.409892056000004, 2021-01-15T19:11:07.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:07.831 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.832 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.832 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:07.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.833 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:07.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:07.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:07.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:07.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:07.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:07.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:07.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:07.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:07.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:07.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:07.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:07.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:07.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:07.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:07.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:07.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:07.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:07.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:07.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:07.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:07.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:07.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:07.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:07.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:07.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:07.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:07.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:07.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:07.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:07.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:07.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:07.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.849 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:07.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:07.850 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.406756539, 2021-01-15T19:11:07.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:07.850 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.851 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.851 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:07.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.852 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:07.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:07.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:07.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:07.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:07.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:07.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:07.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:07.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:07.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:07.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:07.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:07.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:07.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:07.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:07.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:07.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:07.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:07.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:07.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:07.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:07.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:07.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:07.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:07.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:07.866 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.866 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.866 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.867 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:07.867 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:07.867 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 89.406652512999997, 2021-01-15T19:11:07.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:07.867 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.868 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.868 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.868 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:07.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.869 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:07.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:07.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:07.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:07.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:07.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:07.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:07.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:07.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:07.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:07.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:07.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:07.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:07.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:07.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:07.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:07.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:07.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:07.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:07.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:07.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:07.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:07.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:07.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:07.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:07.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:07.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:07.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:07.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:07.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:07.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:07.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:07.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.884 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.885 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:07.885 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:07.885 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.395665764, 2021-01-15T19:11:07.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:07.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.887 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:07.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.887 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:07.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:07.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:07.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:07.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:07.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:07.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:07.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:07.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:07.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:07.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:07.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:07.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:07.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:07.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:07.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:07.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:07.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:07.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:07.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:07.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:07.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:07.898 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.898 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.898 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.899 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:07.899 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:07.899 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.395570242999995, 2021-01-15T19:11:07.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:07.900 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.900 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.901 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.901 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:07.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.901 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:07.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:07.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:07.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:07.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:07.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:07.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:07.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:07.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:07.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:07.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:07.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:07.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:07.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:07.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:07.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:07.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:07.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:07.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:07.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:07.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:07.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:07.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.914 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:07.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:07.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.395459097, 2021-01-15T19:11:07.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:07.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.916 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:07.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:07.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:07.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:07.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:07.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:07.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:07.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:07.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:07.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:07.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:07.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:07.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:07.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:07.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:07.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:07.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:07.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:07.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:07.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:07.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:07.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:07.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:07.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:07.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:07.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.303684775000001, 2021-01-15T19:11:07.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:07.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.930 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:07.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.931 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:07.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:07.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:07.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:07.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:07.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:07.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:07.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:07.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:07.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:07.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:07.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:07.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:07.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:07.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:07.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:07.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:07.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:07.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:07.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:07.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.943 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.944 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:07.944 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:07.944 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.302401635999999, 2021-01-15T19:11:07.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:07.944 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.945 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:07.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.945 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.945 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:07.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.946 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:07.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:07.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:07.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:07.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:07.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:07.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:07.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:07.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:07.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:07.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:07.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:07.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:07.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:07.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:07.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:07.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:07.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:07.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:07.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:07.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:07.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:07.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:07.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:07.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:07.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:07.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.958 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.958 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.959 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:07.959 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:07.959 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.397273668, 2021-01-15T19:11:07.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:07.960 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:07.960 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:07.960 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:07.960 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:07.960 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:07.961 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:07.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.961 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:07.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:07.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:07.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:07.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:07.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:07.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:07.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:07.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:07.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:07.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:07.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:07.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:07.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:07.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:07.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:07.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:07.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:07.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:07.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:07.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:07.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:07.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:07.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:07.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:07.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:07.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:07.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:07.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:07.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:07.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.970 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:07.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:07.971 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:07.973 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:07.973 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:07.973 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:07.974 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:07.974 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:07.974 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:07.974 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:07.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:07.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:07.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:07.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:07.975 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:07.976 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:07.976 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:07.976 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:07.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:07.977 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:07.971+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:08.074 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:08.072+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:08.141 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:08.141 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:08.142 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:08.142 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:08.151 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:08.192 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:08.190+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:08.276 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:08.274+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:08.320 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:08.320 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:08.320 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:08.321 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:08.321 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:08.329 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:08.414 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:08.412+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:08.477 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:08.475+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:08.497 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:08.497 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:11:08.498 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.208840431, 2021-01-15T19:11:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:11:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.499 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:11:08.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:08.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:08.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:11:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:11:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:11:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:11:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:11:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:11:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:11:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:11:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:11:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:11:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:11:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:11:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:11:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:11:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:11:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:11:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:11:08.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:11:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:11:08.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:11:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:11:08.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:11:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:11:08.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:11:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:11:08.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.27707766500001, 2021-01-15T19:11:08.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:11:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:11:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:11:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:11:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:11:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:11:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:11:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:11:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:11:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:11:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:11:08.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:11:08.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:11:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:11:08.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:11:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:11:08.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:11:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:11:08.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:11:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:11:08.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:11:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:11:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:11:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:11:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:11:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:11:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:11:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:08.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.27478962399999, 2021-01-15T19:11:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:11:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.536 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:11:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.537 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:08.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:08.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:11:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:11:08.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:11:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:11:08.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:11:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:11:08.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:11:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:11:08.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:11:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:11:08.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:11:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:11:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:11:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:11:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:11:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:11:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:11:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:11:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:11:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:11:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:11:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.550 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:08.551 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.27219999900001, 2021-01-15T19:11:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:11:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.552 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:11:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:08.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:08.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:11:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:11:08.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:11:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:11:08.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:11:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:11:08.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:11:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:11:08.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:11:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:11:08.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:11:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:11:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:11:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:11:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:11:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:11:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:11:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:11:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:11:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:11:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:11:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:11:08.567 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 159.27129172100001, 2021-01-15T19:11:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:11:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.568 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:11:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.569 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:11:08.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:11:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:11:08.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:11:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:11:08.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:11:08.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:11:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:11:08.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:11:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:11:08.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:11:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:11:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:11:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:11:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:11:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:11:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:11:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:11:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:11:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.302607814, 2021-01-15T19:11:08.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:11:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:11:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.584 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:11:08.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:11:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:11:08.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:11:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:11:08.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:11:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:11:08.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:11:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:11:08.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:11:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:11:08.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:11:08.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:11:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:11:08.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:11:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:11:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:11:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:11:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:11:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:11:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:11:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:11:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:11:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:11:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:11:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:08.600 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.30245807700001, 2021-01-15T19:11:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:11:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:11:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.602 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:08.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:11:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:08.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:11:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:11:08.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:11:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:11:08.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:11:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:11:08.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:11:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:11:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:11:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:11:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:11:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:11:08.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:08.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:11:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:11:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:11:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:11:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:08.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:11:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:11:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:11:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:11:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:11:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:11:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:11:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:11:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.302306179, 2021-01-15T19:11:08.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:11:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:11:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:11:08.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:11:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:11:08.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:11:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:11:08.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:11:08.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:11:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:11:08.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:11:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:11:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:11:08.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:11:08.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:11:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:11:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:11:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:11:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:11:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:11:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:11:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:11:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:11:08.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:11:08.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:11:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:11:08.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:11:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 155.302180867, 2021-01-15T19:11:08.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:11:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:11:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:08.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:08.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:11:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:11:08.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:11:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:11:08.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:11:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:11:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:11:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:11:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:11:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:11:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:11:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:08.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:11:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:11:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:11:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:11:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:11:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:11:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:11:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:11:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:11:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:11:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:11:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:11:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:11:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:11:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:11:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:11:08.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:11:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.28134829199999, 2021-01-15T19:11:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:11:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.658 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:11:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:11:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:11:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:11:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:11:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:11:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:11:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:11:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:11:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:11:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:11:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:11:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:11:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:11:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:11:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:11:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:11:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:11:08.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:11:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:11:08.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:11:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:11:08.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:11:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 153.27505496000001, 2021-01-15T19:11:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:11:08.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:11:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:11:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:11:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:11:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:11:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:11:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:11:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:11:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:11:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:11:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:11:08.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:11:08.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:11:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:11:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:11:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:11:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:11:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:11:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:11:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:11:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.687 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:11:08.688 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.31299544699999, 2021-01-15T19:11:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:11:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.689 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:11:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:11:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:11:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:11:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:11:08.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:11:08.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:11:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:11:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:11:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:11:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:11:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:11:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:11:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:11:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:11:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:11:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:11:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:11:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:11:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:11:08.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:11:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:11:08.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:11:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:11:08.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:11:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:11:08.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.706 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.250623467, 2021-01-15T19:11:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:11:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.708 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:11:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:11:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:11:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:11:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:11:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:11:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:11:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:11:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:11:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:11:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:11:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:11:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:11:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:11:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:11:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:11:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:11:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:11:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:11:08.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:11:08.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:11:08.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:11:08.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.721 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.722 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.24732096599999, 2021-01-15T19:11:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:11:08.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.724 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:11:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:11:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:11:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:11:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:11:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:11:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:11:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:11:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:11:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:11:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:11:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:11:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:11:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:11:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:11:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:11:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:11:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:11:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:11:08.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:11:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.24599910000001, 2021-01-15T19:11:08.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:11:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:11:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:11:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:11:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:11:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:11:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:11:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:11:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:11:08.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:11:08.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:11:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:11:08.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:11:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:11:08.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:11:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:11:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:11:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:11:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:11:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:11:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:11:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:11:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:08.755 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.24562884700001, 2021-01-15T19:11:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:11:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:11:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:11:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:11:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:11:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:11:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:11:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:11:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:11:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:11:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:11:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:11:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:11:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:11:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:11:08.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:11:08.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:11:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:11:08.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:11:08.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:11:08.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:11:08.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:11:08.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:11:08.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.771 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:08.771 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:08.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 149.245294454, 2021-01-15T19:11:08.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:11:08.772 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.772 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.773 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:11:08.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:08.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:08.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:11:08.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:11:08.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:11:08.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:11:08.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:11:08.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:11:08.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:11:08.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:11:08.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:11:08.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:11:08.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:11:08.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:08.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:11:08.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:11:08.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:11:08.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:11:08.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:11:08.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:11:08.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:11:08.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:11:08.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.785 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.786 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:11:08.786 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:08.786 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 135.30403319499999, 2021-01-15T19:11:08.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:11:08.787 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.787 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.787 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:08.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:08.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:11:08.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:11:08.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:11:08.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:11:08.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:11:08.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:11:08.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:11:08.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:11:08.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:11:08.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:11:08.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:11:08.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:08.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:11:08.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:11:08.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:11:08.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:11:08.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:11:08.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:11:08.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:11:08.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:11:08.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.799 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.800 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:11:08.800 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:08.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.311670323, 2021-01-15T19:11:08.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:11:08.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.801 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:08.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.801 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:11:08.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.802 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:08.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:08.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:11:08.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:11:08.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:11:08.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:11:08.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:11:08.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:11:08.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:11:08.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:11:08.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:11:08.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:11:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:11:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:11:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:11:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:11:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:11:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:11:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:11:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:11:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:11:08.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:11:08.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:11:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:11:08.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:11:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:11:08.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:11:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:11:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.281676119, 2021-01-15T19:11:08.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:11:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:08.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:11:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:11:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:11:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:11:08.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:11:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:11:08.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:11:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:11:08.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:11:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:11:08.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:11:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:11:08.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:11:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:11:08.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:11:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:11:08.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:11:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:11:08.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:11:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:11:08.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:08.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:08.833 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:08.834 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:08.835 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:08.832+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:08.935 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:08.933+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:09.001 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:09.001 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:09.001 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:09.001 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:09.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:09.051 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:09.049+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:09.136 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:09.134+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:09.173 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:09.174 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:09.174 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:09.174 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:09.175 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:09.184 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:09.268 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:09.266+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:09.339 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:09.337+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:09.350 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:09.351 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:11:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:09.352 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 150.163947377, 2021-01-15T19:11:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:11:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:09.353 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:09.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:09.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:11:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:11:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:11:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:11:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:11:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:11:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:11:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:11:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:11:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:11:09.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:11:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:11:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:11:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:11:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:11:09.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:11:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:11:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:09.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 146.11645920000001, 2021-01-15T19:11:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:11:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:09.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:11:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.369 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:09.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:09.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:11:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:11:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:11:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:11:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:11:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:11:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:11:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:11:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:11:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:11:09.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:11:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:11:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:11:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:11:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:11:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:11:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:11:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:11:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:11:09.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:11:09.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:11:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:11:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 146.116311182, 2021-01-15T19:11:09.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:11:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:11:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:09.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:11:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:11:09.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:11:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:11:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:11:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:11:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:11:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:11:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:11:09.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:11:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:11:09.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:11:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:11:09.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:11:09.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:11:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:11:09.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:11:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:11:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:11:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:11:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:11:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:11:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:11:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.402 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:09.403 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.161990191, 2021-01-15T19:11:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:11:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.405 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:11:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:11:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:11:09.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:11:09.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:11:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:11:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:11:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:11:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:11:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:11:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:11:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:11:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:11:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:11:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:11:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:11:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:11:09.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:11:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:11:09.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.419 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.161798902, 2021-01-15T19:11:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:11:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.420 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.421 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:11:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:11:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:11:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:11:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:11:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:11:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:11:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:11:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:11:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:11:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:11:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:11:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:11:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:11:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:11:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:11:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:11:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:11:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:11:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:09.434 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 111.16163654899999, 2021-01-15T19:11:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:11:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.435 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:09.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:09.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:09.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:11:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:11:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:11:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:11:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:11:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:11:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:11:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:11:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:11:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:11:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:11:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:11:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:11:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:11:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:11:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:11:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:11:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:11:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:11:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:09.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 105.129916636, 2021-01-15T19:11:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:11:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.450 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:11:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.451 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:11:09.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:11:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:11:09.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:11:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:11:09.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:11:09.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:11:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:11:09.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:11:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:11:09.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:11:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:11:09.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:11:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:11:09.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:11:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 105.129854947, 2021-01-15T19:11:09.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:11:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:11:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:11:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:11:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:11:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:11:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:11:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:11:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:11:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:11:09.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:11:09.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:11:09.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:11:09.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:11:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:11:09.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:11:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:11:09.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.474 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:11:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.025788609000003, 2021-01-15T19:11:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:11:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.475 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.476 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:11:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:09.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:11:09.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:11:09.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:11:09.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:11:09.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:11:09.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:11:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:11:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:11:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:11:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:11:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:11:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:11:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:11:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:11:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:11:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:11:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:11:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:11:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:11:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:11:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:11:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:11:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:11:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:11:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.491 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:11:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.025702037000002, 2021-01-15T19:11:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:11:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:11:09.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:09.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:09.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:11:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:09.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:11:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:11:09.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:11:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:11:09.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:11:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:11:09.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:11:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:11:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:11:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:09.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:11:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:11:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:11:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:11:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:09.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:11:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:11:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:11:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:11:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:09.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:11:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:11:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:11:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:11:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:11:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:11:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:11:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:11:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:11:09.511 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 66.135391908000003, 2021-01-15T19:11:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:11:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:09.512 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:11:09.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:11:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:11:09.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:11:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:11:09.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:11:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:11:09.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:11:09.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:11:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:11:09.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:11:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:11:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:11:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:11:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:11:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:11:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:11:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:11:09.524 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.010231058999999, 2021-01-15T19:11:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:11:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.526 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:11:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:11:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:11:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:11:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:11:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:11:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:11:09.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:11:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:11:09.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:11:09.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:11:09.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:11:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:11:09.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:11:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:11:09.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:11:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:11:09.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:11:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:11:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:11:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:11:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.010135138999999, 2021-01-15T19:11:09.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:11:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.542 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:11:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:11:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:11:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:11:09.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:11:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:11:09.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:11:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:11:09.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:11:09.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:11:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:11:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:11:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:09.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:11:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:11:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:11:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:11:09.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:11:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:11:09.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:11:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:11:09.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:11:09.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:11:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:11:09.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:11:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:11:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.143076798999999, 2021-01-15T19:11:09.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:11:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.560 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:11:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:11:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:11:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:11:09.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:11:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:11:09.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:11:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:11:09.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:11:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:11:09.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:09.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:11:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:11:09.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:11:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:11:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:11:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:11:09.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:11:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:11:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:11:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:11:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:11:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:09.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:11:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:11:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:11:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:11:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:11:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:09.577 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.142985721999999, 2021-01-15T19:11:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:11:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.578 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:11:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:11:09.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:11:09.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:11:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:11:09.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:11:09.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:11:09.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:11:09.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:11:09.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:11:09.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:11:09.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:09.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:11:09.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:11:09.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:11:09.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:11:09.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:09.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:11:09.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:11:09.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:11:09.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:11:09.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:09.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:11:09.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:11:09.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:11:09.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:11:09.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:11:09.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:11:09.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:11:09.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:11:09.595 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.596 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.596 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:11:09.597 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:09.597 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.133506467, 2021-01-15T19:11:09.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:11:09.597 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.598 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:09.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.599 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:11:09.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.599 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:09.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:09.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:11:09.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:11:09.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:11:09.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:11:09.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:11:09.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:11:09.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:11:09.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:11:09.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:11:09.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:11:09.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:11:09.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:09.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:11:09.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:11:09.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:11:09.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:11:09.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:11:09.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:11:09.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:09.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:11:09.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:11:09.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:09.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:11:09.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:11:09.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:09.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:11:09.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:11:09.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:11:09.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:11:09.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:11:09.616 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:09.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.133361442999998, 2021-01-15T19:11:09.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:11:09.617 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.617 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.618 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:11:09.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.619 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:09.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:09.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:11:09.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:11:09.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:11:09.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:11:09.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:11:09.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:11:09.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:11:09.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:11:09.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:11:09.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:11:09.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:11:09.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:09.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:11:09.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:11:09.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:11:09.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:11:09.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:09.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:11:09.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:11:09.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:11:09.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:11:09.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:09.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:11:09.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:11:09.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:11:09.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:11:09.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:11:09.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:11:09.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:11:09.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:11:09.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.636 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.637 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:09.637 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:09.638 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.967867569999999, 2021-01-15T19:11:09.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:11:09.638 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.638 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:11:09.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:09.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:09.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:11:09.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:11:09.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:11:09.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:11:09.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:11:09.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:11:09.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:11:09.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:11:09.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:11:09.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:11:09.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:11:09.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:11:09.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:11:09.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:11:09.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:11:09.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.651 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.652 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:09.652 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:09.652 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.967780142999999, 2021-01-15T19:11:09.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:11:09.653 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.653 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.654 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:11:09.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:09.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:09.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:11:09.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:11:09.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:11:09.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:11:09.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:11:09.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:11:09.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:11:09.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:11:09.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:11:09.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:11:09.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:11:09.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:11:09.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:11:09.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:11:09.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:11:09.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:09.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:09.666 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.967700179000001, 2021-01-15T19:11:09.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:11:09.666 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:09.667 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:09.667 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:09.667 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:09.667 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:09.667 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:11:09.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.668 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:09.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:09.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:09.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:09.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:09.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:09.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:11:09.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:09.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:11:09.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:11:09.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:09.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:11:09.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:11:09.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:09.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:11:09.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:11:09.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:09.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:11:09.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:11:09.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:09.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:11:09.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:11:09.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:09.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:11:09.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:11:09.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:09.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:09.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:09.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:11:09.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:11:09.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.677 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:09.678 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:09.678 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:09.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:11:09.681 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:09.681 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:09.679+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:09.781 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:09.780+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:09.845 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:09.845 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:09.846 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:09.846 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:09.855 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:09.900 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:09.898+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:09.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:09.981 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:09.981 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:09.982 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:09.982 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:09.982 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:09.985 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:09.983+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:09.991 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:10.086 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:10.084+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:10.126 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:10.146 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:10.159 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:10.160 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:10.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:10.160 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:11:10.160 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:10.166 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:10.170 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:10.187 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:10.208 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:10.211 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:10.210+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:10.229 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:10.231 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:10.231 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:10.231 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:10.232 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:10.233 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:10.288 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:10.287+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:10.303 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:10.303 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:10.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:11:10.388 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:10.387+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:10.446 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:10.447 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:10.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:10.489 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:10.488+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:10.588 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:10.588 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:10.591 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:10.590+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:10.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:10.691 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:10.690+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:10.770 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:10.771 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:10.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:10.792 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:10.791+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:10.893 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:10.892+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:10.911 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:10.911 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:10.921 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:11.005 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:11.004+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:11.093 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.093 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:11.095 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:11.094+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:11.103 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:11.196 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:11.195+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:11.233 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.234 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:11.243 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:11.327 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:11.326+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:11.348 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:11.349 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:11.349 INFO:tasks.thrashosds.thrasher:inject_pause on 5 2021-01-15T19:11:11.349 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2021-01-15T19:11:11.350 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2021-01-15T19:11:11.350 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2021-01-15T19:11:11.398 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:11.397+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:11.412 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.412 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:11.422 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:11.479 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:11.479 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2021-01-15T19:11:11.480 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:11.499 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:11.498+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:11.588 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.589 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:11.598 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:11.599 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:11.598+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:11.701 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:11.700+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:11.724 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.724 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:11.733 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:11.802 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:11.801+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:11.867 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:11.911 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:11.910+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:11.991 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:11.992 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:12.002 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:12.006 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:12.005+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:12.107 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:12.106+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:12.165 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:12.165 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:12.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:12.220 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:12.219+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:12.235 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:12.299 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:12.300 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:12.308 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:12.307+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:12.310 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:12.360 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:11:12.405 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:12.409 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:12.408+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:12.425 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:12.446 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:12.467 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:12.475 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:12.475 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:12.485 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:12.488 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:12.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:12.509 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:12.509 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:12.509 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:12.510 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:12.510 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:12.510 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:12.510 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:12.510 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:12.511 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:12.511 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:12.512 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:12.512 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:12.528 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:12.527+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:12.610 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:12.610 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:12.611 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:12.611 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:12.613 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:12.612+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:12.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:12.714 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:12.713+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:12.785 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:12.786 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:12.786 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:12.786 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:12.786 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:12.795 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:12.836 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:12.835+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:12.915 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:12.914+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:12.958 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:12.959 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:12.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:12.959 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:11:12.959 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:12.969 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:13.016 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:13.015+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:13.103 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:13.103 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:13.103 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:13.104 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:13.113 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:13.156 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:13.155+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:13.213 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:11:13.218 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:13.217+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:13.248 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:13.248 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:13.249 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:13.249 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:13.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:13.319 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:13.318+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:13.419 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:13.418+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:13.434 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:13.434 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:13.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:13.434 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:13.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.435 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:13.435 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:13.435 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.99597128600001, 2021-01-15T19:11:13.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:11:13.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.436 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:11:13.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.437 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:13.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:13.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:11:13.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:11:13.440 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:11:13.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:11:13.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:11:13.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.442 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:11:13.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:13.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.443 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.443 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:11:13.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:11:13.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.444 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.444 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:11:13.444 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:11:13.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.445 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.445 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:11:13.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:11:13.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.446 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.446 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:11:13.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:11:13.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:11:13.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:11:13.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:11:13.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:11:13.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:11:13.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:11:13.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:11:13.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:11:13.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:11:13.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:11:13.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:11:13.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:11:13.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:11:13.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:11:13.454 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.455 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.456 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:13.456 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:13.456 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.99583198600001, 2021-01-15T19:11:13.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:11:13.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.457 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.457 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.457 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:11:13.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.458 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.458 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.458 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:13.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.459 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.459 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:13.460 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:11:13.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.460 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.460 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.460 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:11:13.461 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:11:13.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.461 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:11:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:11:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:11:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:11:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:11:13.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:11:13.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:11:13.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:11:13.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:11:13.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:11:13.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:11:13.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:11:13.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:11:13.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:11:13.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:11:13.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:11:13.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.470 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:11:13.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:11:13.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:11:13.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:11:13.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:11:13.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:11:13.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:11:13.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:11:13.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:11:13.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:11:13.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.475 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.476 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:13.476 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:13.476 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.99571087999999, 2021-01-15T19:11:13.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:11:13.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:11:13.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.479 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:13.479 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:13.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:11:13.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:11:13.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:11:13.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:11:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:11:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:11:13.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:11:13.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:11:13.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:11:13.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:11:13.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:11:13.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.487 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:11:13.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:11:13.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:11:13.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:11:13.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:11:13.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:11:13.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.490 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:11:13.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:11:13.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:11:13.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:11:13.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:11:13.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:11:13.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:11:13.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:11:13.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.494 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.494 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:11:13.494 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:11:13.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.495 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.495 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.495 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:11:13.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:11:13.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.496 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.496 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.496 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.497 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:13.497 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:13.497 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.22543000600001, 2021-01-15T19:11:13.497 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:11:13.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.498 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.499 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:11:13.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.499 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:13.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:13.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:11:13.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:11:13.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:11:13.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:11:13.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:11:13.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:11:13.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:11:13.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:11:13.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:11:13.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:11:13.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:11:13.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:11:13.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:11:13.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:11:13.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:11:13.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:11:13.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:11:13.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:11:13.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:11:13.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.511 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.511 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:11:13.511 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:11:13.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.512 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.512 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.512 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:11:13.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:11:13.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.513 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.513 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.513 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:11:13.513 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:11:13.513 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.514 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.515 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:13.515 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:13.515 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.216451828, 2021-01-15T19:11:13.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:11:13.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.516 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:11:13.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.517 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:13.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:13.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:11:13.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:11:13.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:11:13.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:11:13.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:11:13.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:11:13.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:13.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:11:13.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:11:13.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:13.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:11:13.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.525 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:11:13.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:11:13.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:11:13.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:11:13.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:11:13.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:11:13.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:11:13.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:11:13.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:11:13.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:11:13.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.530 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:11:13.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:11:13.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:11:13.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:11:13.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.532 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.532 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.533 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:13.533 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:13.533 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.21593212799999, 2021-01-15T19:11:13.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:11:13.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.534 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:11:13.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.535 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:13.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:13.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:11:13.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:11:13.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:11:13.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:11:13.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:11:13.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:11:13.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:11:13.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:11:13.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:11:13.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:11:13.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:11:13.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:11:13.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:11:13.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:11:13.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:11:13.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:11:13.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:11:13.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:11:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:11:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:11:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:11:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:11:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:11:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:11:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:11:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.214251174, 2021-01-15T19:11:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:11:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:11:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:11:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:11:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:11:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:11:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:11:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:11:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:11:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:11:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:11:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:11:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:11:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:11:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:11:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:11:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:11:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:11:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:11:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:11:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:11:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:11:13.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.566 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:11:13.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:11:13.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:11:13.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:11:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 149.21411821999999, 2021-01-15T19:11:13.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:11:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.570 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:11:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:11:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:11:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:11:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.574 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:11:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:11:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:11:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:11:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:11:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:11:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:11:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:11:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.578 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:11:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:11:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:11:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:11:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:11:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:11:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:11:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:11:13.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:11:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:11:13.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:11:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:11:13.584 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.585 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:11:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 140.25158694999999, 2021-01-15T19:11:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:11:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:11:13.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.588 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:13.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:13.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:11:13.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.590 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:11:13.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:11:13.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:11:13.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:11:13.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.592 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:11:13.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:11:13.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:11:13.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:11:13.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.594 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.594 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:11:13.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:11:13.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.595 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:13.595 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:11:13.596 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:11:13.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.596 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.596 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.596 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:11:13.597 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:11:13.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.597 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:11:13.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:11:13.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.598 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:11:13.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:11:13.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:11:13.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:11:13.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:11:13.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:11:13.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.601 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:11:13.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:11:13.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.602 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.603 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:11:13.603 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:13.603 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 140.251420357, 2021-01-15T19:11:13.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:11:13.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:13.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.604 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:11:13.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.605 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.606 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:13.606 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.607 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:13.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:11:13.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:11:13.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:11:13.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:11:13.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:11:13.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:11:13.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:11:13.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.611 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.611 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:11:13.611 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:11:13.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.612 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.612 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:11:13.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:11:13.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.613 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:13.613 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:11:13.613 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:11:13.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.614 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.614 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:11:13.614 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:11:13.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:13.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:11:13.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:11:13.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.616 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:11:13.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:11:13.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:13.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:11:13.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:11:13.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:11:13.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:11:13.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:11:13.620 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:11:13.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:11:13.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:11:13.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.621 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.622 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:13.622 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:13.623 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.052881563, 2021-01-15T19:11:13.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:11:13.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.623 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.624 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:11:13.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.624 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.625 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:13.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:13.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:11:13.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:11:13.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:11:13.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:11:13.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:11:13.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.629 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.629 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:11:13.629 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:11:13.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.630 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.630 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.630 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:11:13.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:11:13.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.631 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.631 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:11:13.631 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:11:13.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.632 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.632 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:11:13.632 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:11:13.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:11:13.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:11:13.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:11:13.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:11:13.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:11:13.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:11:13.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.636 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:11:13.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:11:13.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.637 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:11:13.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:11:13.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:11:13.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:11:13.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.639 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.640 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:13.640 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:13.640 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.050611518, 2021-01-15T19:11:13.640 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:11:13.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.641 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.642 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:11:13.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.642 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:13.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:13.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:11:13.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:11:13.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:11:13.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:11:13.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:11:13.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:11:13.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:11:13.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:11:13.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:11:13.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:11:13.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:11:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:11:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:11:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:11:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:11:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:11:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:11:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:11:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:11:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:11:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:11:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:11:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:11:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:11:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:11:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:13.658 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.046073327, 2021-01-15T19:11:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:11:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.659 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:11:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:13.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:11:13.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:11:13.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:11:13.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:11:13.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:11:13.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.664 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:11:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:11:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:11:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:11:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:11:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:11:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:11:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:11:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:11:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:11:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:11:13.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:11:13.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:11:13.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:11:13.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:11:13.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:11:13.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:11:13.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:11:13.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:11:13.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.676 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:13.676 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:13.677 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.043487455, 2021-01-15T19:11:13.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:11:13.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.677 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.678 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:11:13.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.679 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:13.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:13.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:11:13.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:11:13.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:11:13.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:11:13.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:11:13.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:11:13.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:11:13.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:11:13.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:11:13.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:11:13.686 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:11:13.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.686 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.686 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:11:13.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:11:13.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.687 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.687 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:11:13.688 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:11:13.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.688 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.688 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.688 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:11:13.689 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:11:13.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.690 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:11:13.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:11:13.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:11:13.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:11:13.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:11:13.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:11:13.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:11:13.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:11:13.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.694 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:13.694 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:13.695 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.04311312, 2021-01-15T19:11:13.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:11:13.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.695 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.696 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:11:13.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.696 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:13.697 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:13.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:11:13.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.699 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:11:13.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:11:13.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:11:13.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:11:13.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:11:13.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:11:13.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.702 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:11:13.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:11:13.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:11:13.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:11:13.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:11:13.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:11:13.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:11:13.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:11:13.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:11:13.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:11:13.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.707 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.707 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:11:13.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:11:13.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.708 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.708 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:11:13.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:11:13.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.709 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.709 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:11:13.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:11:13.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:11:13.710 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:11:13.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.711 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.711 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:13.712 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:13.712 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 121.04297662499999, 2021-01-15T19:11:13.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:11:13.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:13.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.713 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:11:13.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.714 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:13.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:13.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:11:13.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:11:13.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:11:13.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:11:13.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:11:13.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:11:13.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:11:13.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:11:13.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:11:13.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:11:13.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:11:13.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:11:13.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:11:13.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:11:13.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:11:13.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:13.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:11:13.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:11:13.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:11:13.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:11:13.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.725 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.725 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:11:13.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:11:13.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:11:13.727 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:11:13.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.728 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:11:13.728 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:13.728 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 119.060578905, 2021-01-15T19:11:13.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:11:13.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:13.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.729 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:11:13.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.730 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.731 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:13.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.731 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:13.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:11:13.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:11:13.733 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:11:13.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:11:13.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:11:13.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:11:13.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:11:13.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.736 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.736 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:11:13.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:11:13.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:11:13.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:11:13.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:13.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:11:13.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:11:13.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:11:13.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:11:13.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:13.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:11:13.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:11:13.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:11:13.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:11:13.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:11:13.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:11:13.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:11:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:11:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.744 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:11:13.745 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:13.745 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.913825846999998, 2021-01-15T19:11:13.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:11:13.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:13.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.746 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:11:13.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.747 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:13.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.748 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:13.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:11:13.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:11:13.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:11:13.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:11:13.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:11:13.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:11:13.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:11:13.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.752 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:11:13.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:11:13.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:11:13.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:11:13.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.754 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:11:13.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:11:13.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:13.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:11:13.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:11:13.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:13.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:11:13.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:11:13.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:13.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:11:13.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:11:13.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:11:13.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:11:13.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.759 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.760 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:11:13.760 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:13.760 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 51.913662915000003, 2021-01-15T19:11:13.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:11:13.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:13.761 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:13.761 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:13.761 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:13.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:11:13.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.762 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:13.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:13.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:11:13.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.765 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:11:13.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:11:13.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.765 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.766 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:11:13.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:11:13.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.766 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.767 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:11:13.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:11:13.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.767 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.768 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:11:13.768 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:11:13.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.768 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:11:13.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:11:13.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:13.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:11:13.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:11:13.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:11:13.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:11:13.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:13.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:11:13.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:11:13.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:11:13.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:11:13.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:11:13.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:11:13.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:11:13.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:11:13.775 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.775 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:11:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 46.139990277999999, 2021-01-15T19:11:13.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:11:13.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:13.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:13.777 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:13.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:13.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:13.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:13.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:13.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:13.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:11:13.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:13.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:11:13.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:11:13.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:13.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:11:13.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:11:13.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:13.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:11:13.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:11:13.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:13.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:11:13.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:11:13.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:11:13.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:11:13.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:13.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:11:13.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:11:13.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:13.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:11:13.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:11:13.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:13.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:11:13.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:11:13.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:13.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:13.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:13.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:11:13.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:11:13.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:13.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:13.791 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:13.790+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:13.791 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:13.892 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:13.891+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:13.928 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:13.928 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:13.929 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:13.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:13.992 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:13.991+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:14.069 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:14.069 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:14.070 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:14.070 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:14.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:14.094 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:14.093+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:14.195 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:14.194+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:14.252 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:14.252 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:14.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:14.253 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:14.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.253 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:14.253 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:14.253 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 155.00078287599999, 2021-01-15T19:11:14.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:11:14.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.254 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.255 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:11:14.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.255 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:14.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:14.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:11:14.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:11:14.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:11:14.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.259 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.259 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:11:14.259 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:11:14.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.260 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.260 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.260 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:11:14.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:11:14.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.261 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.261 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.261 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:11:14.262 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:11:14.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:11:14.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:11:14.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:11:14.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:11:14.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:11:14.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:11:14.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:11:14.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:11:14.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:11:14.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:11:14.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:11:14.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:11:14.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.268 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.269 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:14.269 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:14.270 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 155.00059193199999, 2021-01-15T19:11:14.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:11:14.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:14.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.272 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.273 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:11:14.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.273 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:14.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:14.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:11:14.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.276 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.276 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:11:14.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:11:14.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.276 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.277 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.277 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:11:14.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:11:14.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.278 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.278 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.278 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:11:14.278 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:11:14.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.279 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:11:14.279 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:11:14.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:11:14.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:11:14.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.281 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:14.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:11:14.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:11:14.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:11:14.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:11:14.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:14.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:11:14.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:11:14.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:11:14.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:11:14.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:11:14.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:11:14.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.286 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:11:14.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:11:14.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.288 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.288 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:14.288 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.06163983499999, 2021-01-15T19:11:14.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:11:14.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.289 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.290 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:11:14.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.290 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:14.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:14.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:11:14.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:11:14.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:11:14.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.293 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.294 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.294 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:11:14.294 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:11:14.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.295 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.295 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:11:14.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:11:14.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.296 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.296 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:11:14.296 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:11:14.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:11:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:11:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:11:14.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:11:14.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:11:14.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:11:14.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:11:14.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:11:14.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:11:14.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:11:14.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:11:14.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:11:14.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.303 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.304 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:14.304 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:14.304 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.058919374, 2021-01-15T19:11:14.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:11:14.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:14.305 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.305 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.305 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:11:14.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.306 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:14.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:14.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:11:14.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:11:14.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:11:14.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.310 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:11:14.310 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:11:14.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.310 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.311 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:11:14.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:11:14.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.311 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.312 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:11:14.312 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:11:14.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.312 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.313 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:11:14.313 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:11:14.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:14.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:11:14.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:11:14.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:11:14.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:11:14.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:14.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:11:14.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:11:14.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:11:14.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:11:14.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:11:14.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:11:14.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:11:14.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:11:14.319 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.320 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.320 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.321 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.321 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:14.321 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.05880679500001, 2021-01-15T19:11:14.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:11:14.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.322 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:11:14.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.323 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:14.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:14.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:11:14.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:11:14.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:11:14.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.326 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.326 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.327 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:11:14.327 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:11:14.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.327 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.328 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:11:14.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:11:14.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.328 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.328 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.329 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:11:14.329 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:11:14.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:11:14.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:11:14.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:11:14.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:11:14.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:11:14.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:11:14.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:11:14.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:11:14.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:11:14.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:11:14.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:11:14.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:11:14.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.336 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.336 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.337 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:14.337 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:14.337 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.058676891, 2021-01-15T19:11:14.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:11:14.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.338 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:14.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.338 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:11:14.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.339 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:14.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:14.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:11:14.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:11:14.342 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:11:14.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.342 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.342 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:11:14.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:11:14.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.343 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.343 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:11:14.344 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:11:14.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.344 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:11:14.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:11:14.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:11:14.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:11:14.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:14.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:11:14.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:11:14.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:11:14.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:11:14.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:14.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:11:14.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:11:14.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:11:14.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:11:14.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:11:14.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:11:14.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:11:14.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:11:14.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.352 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.353 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.354 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:14.354 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.03547162199999, 2021-01-15T19:11:14.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:11:14.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.355 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:11:14.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.356 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:14.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:14.358 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:11:14.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.358 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.358 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:11:14.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:11:14.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.359 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.359 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:11:14.360 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:11:14.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:11:14.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:11:14.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:11:14.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:11:14.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:11:14.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:11:14.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:14.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:11:14.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:11:14.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:11:14.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:11:14.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:11:14.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:11:14.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:11:14.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:11:14.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:11:14.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.369 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.369 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:14.370 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.03535342399999, 2021-01-15T19:11:14.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:11:14.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.370 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.371 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:11:14.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.371 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.372 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.372 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:14.372 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.373 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.373 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:14.374 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:11:14.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.374 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.374 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:11:14.375 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:11:14.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.375 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.375 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:11:14.376 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:11:14.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.376 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.377 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:11:14.377 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:11:14.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.377 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:11:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:11:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.378 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.379 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:11:14.379 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:11:14.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.379 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.380 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:11:14.380 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:11:14.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.380 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.381 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:11:14.381 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:11:14.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.381 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.382 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:11:14.382 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:11:14.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.383 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.383 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:11:14.383 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:11:14.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.384 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.384 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.384 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:11:14.384 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:11:14.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.385 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.385 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.385 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.385 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.386 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:14.386 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.03524073400001, 2021-01-15T19:11:14.386 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:11:14.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.387 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:11:14.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.388 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.388 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.388 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:14.388 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.389 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.389 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.389 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:14.389 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:11:14.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.390 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.390 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.390 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:11:14.390 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:11:14.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.391 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.391 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:11:14.391 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:11:14.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.392 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.392 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:11:14.393 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:11:14.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.393 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.393 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.393 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:11:14.394 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:11:14.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.394 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.394 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.394 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:11:14.395 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:11:14.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.395 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.395 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:11:14.396 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:11:14.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.396 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.396 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.396 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:11:14.397 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:11:14.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.397 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.397 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.397 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:11:14.398 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:11:14.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.398 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.398 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.398 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:11:14.399 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:11:14.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.399 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.399 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.399 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:11:14.400 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:11:14.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.400 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.401 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:11:14.401 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:14.401 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.029638984, 2021-01-15T19:11:14.402 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:11:14.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:11:14.402 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.402 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.403 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.403 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:14.403 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.403 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.404 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.404 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:14.404 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:11:14.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.405 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.405 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:11:14.405 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:11:14.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.405 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.406 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.406 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:11:14.406 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:11:14.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.407 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.407 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.407 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:11:14.407 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:11:14.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.408 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.408 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.408 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:11:14.408 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:11:14.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.409 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.409 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.409 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:11:14.409 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:11:14.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.410 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.410 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.410 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:11:14.411 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:11:14.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.411 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.412 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.412 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.412 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:14.412 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.02117025999999, 2021-01-15T19:11:14.413 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:11:14.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.413 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.414 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:11:14.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.414 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.415 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.415 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.415 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:14.415 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.416 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.416 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.416 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:14.416 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:11:14.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.417 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.417 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.417 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:11:14.417 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:11:14.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.418 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.418 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.418 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:11:14.418 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:11:14.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.419 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.419 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:11:14.419 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:11:14.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.420 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.420 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:11:14.421 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:11:14.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.421 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.421 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.422 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:11:14.422 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:11:14.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.422 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.422 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.423 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:11:14.423 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:11:14.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.423 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.423 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.424 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:11:14.424 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:11:14.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.424 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.425 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:11:14.425 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:11:14.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.425 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.426 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:11:14.426 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:11:14.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.426 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.426 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.427 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:11:14.427 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:11:14.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.427 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.428 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.428 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:14.428 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:14.428 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.016651677, 2021-01-15T19:11:14.429 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:11:14.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.429 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.429 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.430 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:11:14.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.430 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.431 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.431 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:14.431 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.432 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.432 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:14.432 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:11:14.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.433 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.433 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:11:14.433 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:11:14.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.434 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.434 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:11:14.434 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:11:14.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.435 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.435 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:11:14.435 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:11:14.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.436 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.436 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:11:14.436 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:11:14.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.437 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.437 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:11:14.437 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:11:14.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.438 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.438 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.438 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:11:14.438 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:11:14.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.439 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.439 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:11:14.439 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:11:14.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.440 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.440 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.440 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:11:14.441 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:11:14.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.441 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.441 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:11:14.442 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:11:14.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.442 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.442 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.443 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:11:14.443 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:11:14.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.443 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.443 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.444 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:14.444 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:14.444 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 150.01163390799999, 2021-01-15T19:11:14.445 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:11:14.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:14.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.445 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.446 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:11:14.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.446 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.447 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.447 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:14.447 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.448 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.448 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:14.448 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:11:14.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.449 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.449 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:11:14.449 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:11:14.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.450 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.450 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:11:14.450 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:11:14.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.451 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.451 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:11:14.451 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:11:14.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.452 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.452 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:11:14.452 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:11:14.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.453 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.453 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:11:14.453 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:11:14.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.454 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.454 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:14.454 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:11:14.454 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:11:14.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.455 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.455 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:11:14.455 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:11:14.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.456 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.456 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.456 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:11:14.456 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:11:14.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.457 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.457 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:11:14.457 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:11:14.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.458 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.458 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.459 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:11:14.459 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:14.459 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.04450983000001, 2021-01-15T19:11:14.459 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:11:14.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.460 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:11:14.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.461 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.461 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.462 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:14.462 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.462 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.462 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.463 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:14.463 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:11:14.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.463 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.463 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.464 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:11:14.464 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:11:14.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.464 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.465 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:11:14.465 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:11:14.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.465 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.465 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.466 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:11:14.466 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:11:14.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.466 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.466 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.467 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:11:14.467 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:11:14.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.467 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.467 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.468 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:11:14.468 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:11:14.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.468 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:14.469 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:11:14.469 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:11:14.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.469 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.469 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.470 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:11:14.470 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:11:14.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.471 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.471 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:11:14.471 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:11:14.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.472 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.472 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.472 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:11:14.472 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:11:14.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.473 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.473 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:11:14.473 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:11:14.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.474 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.474 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.474 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:11:14.474 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:11:14.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.475 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.475 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.475 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:11:14.475 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:11:14.475 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.476 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.476 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:11:14.477 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:14.477 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 126.067323182, 2021-01-15T19:11:14.477 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:11:14.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.478 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:11:14.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.479 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.479 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.480 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:14.480 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.480 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.480 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.481 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:14.481 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:11:14.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.481 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.481 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.482 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:11:14.482 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:11:14.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.482 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.482 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.483 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:11:14.483 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:11:14.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.483 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.484 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:11:14.484 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:11:14.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.484 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.485 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:11:14.485 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:11:14.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.485 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.485 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:11:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:11:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:14.487 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:11:14.487 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:11:14.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.488 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.488 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:11:14.488 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:11:14.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.488 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.489 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.489 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:11:14.489 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:11:14.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.489 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.490 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.490 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:11:14.490 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:11:14.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.491 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.491 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.491 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:11:14.491 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:11:14.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.492 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.492 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:11:14.492 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:11:14.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.493 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.493 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:11:14.493 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:11:14.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.494 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.495 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:11:14.495 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:14.495 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 76.036776161000006, 2021-01-15T19:11:14.495 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:11:14.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:14.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.496 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:11:14.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.497 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.497 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.498 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:14.498 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.499 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.499 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:14.499 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:11:14.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.499 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.500 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.500 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:11:14.500 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:11:14.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.500 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.501 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.501 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:11:14.501 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:11:14.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.502 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.502 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:11:14.502 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:11:14.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.502 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.503 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.503 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:11:14.503 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:11:14.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.504 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.504 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:11:14.504 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:11:14.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.505 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:14.505 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:11:14.505 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:11:14.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.506 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.506 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:11:14.506 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:11:14.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.507 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:14.507 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:11:14.507 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:11:14.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.508 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.508 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.508 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:11:14.508 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:11:14.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.509 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.509 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.509 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:11:14.509 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:11:14.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.510 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.510 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.510 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:11:14.510 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:11:14.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.511 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.511 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.512 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:14.512 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:14.512 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 71.046118186000001, 2021-01-15T19:11:14.512 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:11:14.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.513 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:11:14.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.514 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.514 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.514 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:14.515 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.515 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.515 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:14.516 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:11:14.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.516 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.516 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.516 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:14.517 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:11:14.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.517 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.518 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:11:14.518 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:11:14.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.518 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.519 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:11:14.519 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:11:14.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.519 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.519 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.520 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:11:14.520 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:11:14.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.520 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.521 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:11:14.521 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:11:14.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.521 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:14.522 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:11:14.522 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:11:14.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.522 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.522 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.523 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:11:14.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:11:14.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.523 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.524 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:11:14.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:11:14.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.524 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.524 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.525 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:11:14.525 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:11:14.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:11:14.526 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:11:14.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:11:14.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:11:14.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:11:14.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:11:14.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.529 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.530 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:14.530 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:14.530 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 71.045978653000006, 2021-01-15T19:11:14.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:11:14.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.531 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:11:14.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.532 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:14.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.533 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:14.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:11:14.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:11:14.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:11:14.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.535 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:11:14.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:11:14.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:11:14.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:11:14.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:11:14.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:11:14.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.539 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.539 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:11:14.539 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:11:14.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.540 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:14.540 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:11:14.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:11:14.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.541 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.541 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:11:14.541 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:11:14.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.541 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:11:14.542 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:11:14.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:11:14.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:11:14.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:11:14.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:11:14.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.545 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:11:14.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:11:14.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.546 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:11:14.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:11:14.547 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.547 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.547 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.548 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:14.548 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:14.548 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 71.045863339999997, 2021-01-15T19:11:14.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:11:14.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:14.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.549 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:11:14.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.550 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.550 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:14.551 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.551 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:14.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:11:14.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.552 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:11:14.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:11:14.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:11:14.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:11:14.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.554 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:11:14.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:11:14.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:14.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:11:14.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:11:14.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:11:14.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:11:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:11:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.559 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.559 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:11:14.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:11:14.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.560 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.560 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:11:14.560 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:11:14.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.561 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:14.561 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:11:14.561 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:11:14.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:14.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:11:14.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:11:14.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:14.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:11:14.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:11:14.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:11:14.564 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:11:14.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.565 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:11:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 34.932392043999997, 2021-01-15T19:11:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:11:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:11:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:14.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:14.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:14.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:14.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:14.570 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:11:14.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:14.571 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:11:14.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:11:14.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:14.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:11:14.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:11:14.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:14.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:11:14.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:11:14.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:14.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:11:14.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:11:14.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.575 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:14.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:11:14.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:11:14.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.576 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:14.576 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:11:14.576 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:11:14.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:14.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:14.577 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:14.577 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:11:14.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:11:14.577 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.578 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:14.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:14.578 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:14.579 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:14.581 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:14.579+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:14.581 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:14.680 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:14.679+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:14.719 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:14.720 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:14.720 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:14.729 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:14.770 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:14.790 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:14.813 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:14.812+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:14.882 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:14.881+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:14.893 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:14.894 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:14.894 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:14.903 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:14.987 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:14.986+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:15.011 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:15.032 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:15.053 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:15.070 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:15.070 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:15.074 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:15.074 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:15.075 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:15.076 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:15.077 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:15.078 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:15.078 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:15.080 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:15.084 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:15.083+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:15.185 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:15.184+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:15.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:11:15.245 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:15.246 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:15.256 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:15.297 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:15.296+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:15.387 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:15.386+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:15.422 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:15.422 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:15.431 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:15.516 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:15.515+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:15.590 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:15.588+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:15.597 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:15.597 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:15.608 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:15.692 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:15.691+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:15.783 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:15.824 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:15.823+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:15.891 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:15.890+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:15.947 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:15.947 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:15.957 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:15.999 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:15.997+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:16.094 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:16.093+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:16.122 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:16.122 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:16.132 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:16.195 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:16.194+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:16.297 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:16.296+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:16.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:11:16.397 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:16.396+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:16.474 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:16.474 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:16.484 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:16.490 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:16.490 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:16.491 INFO:tasks.thrashosds.thrasher:fixing pg num pool device_health_metrics 2021-01-15T19:11:16.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:11:16.536 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:16.535+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:16.598 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:16.597+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:16.649 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:16.649 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:16.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:16.699 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:16.698+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:16.791 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:16.791 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:16.800 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:16.799+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:16.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:11:16.826 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:11:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2021-01-15T19:11:16.355423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276127744,"num_objects":185,"num_object_clones":134,"num_object_copies":553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":10,"num_read":643,"num_read_kb":761253,"num_write":1152,"num_write_kb":281261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":227622912,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3713,"ondisk_log_size":3713,"up":129,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":633536,"kb_used_data":377936,"kb_used_omap":0,"kb_used_meta":255552,"kb_avail":565597504,"statfs":{"total":579820584960,"available":579171844096,"internally_reserved":0,"allocated":387006464,"data_stored":699506767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":261685248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.810002"},"pg_stats":[{"pgid":"3.19","version":"176'70","reported_seq":"313","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.345990+0000","last_change":"2021-01-15T19:10:28.208323+0000","last_active":"2021-01-15T19:11:10.345990+0000","last_peered":"2021-01-15T19:11:10.345990+0000","last_clean":"2021-01-15T19:10:26.286623+0000","last_became_active":"2021-01-15T19:10:28.208323+0000","last_became_peered":"2021-01-15T19:10:28.208323+0000","last_unstale":"2021-01-15T19:11:10.345990+0000","last_undegraded":"2021-01-15T19:10:28.203425+0000","last_fullsized":"2021-01-15T19:10:28.203244+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'51","last_scrub_stamp":"2021-01-15T19:09:27.212108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:27.212108+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":38,"stat_sum":{"num_bytes":9764864,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":16204,"num_write":14,"num_write_kb":4360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2147483647],"acting":[4,7,2147483647],"avail_no_missing":["4(0)","7(1)"],"object_location_counts":[{"shards":"4(0),7(1)","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"174'88","reported_seq":"515","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.345540+0000","last_change":"2021-01-15T19:10:46.280339+0000","last_active":"2021-01-15T19:11:10.345540+0000","last_peered":"2021-01-15T19:11:10.345540+0000","last_clean":"2021-01-15T19:10:44.260389+0000","last_became_active":"2021-01-15T19:10:46.280339+0000","last_became_peered":"2021-01-15T19:10:46.280339+0000","last_unstale":"2021-01-15T19:11:10.345540+0000","last_undegraded":"2021-01-15T19:10:46.271026+0000","last_fullsized":"2021-01-15T19:10:46.270834+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"146'72","last_scrub_stamp":"2021-01-15T19:10:14.380333+0000","last_deep_scrub":"51'34","last_deep_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_clean_scrub_stamp":"2021-01-15T19:10:14.380333+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":1785856,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2235,"num_write":13,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":3047424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2147483647,7],"acting":[4,2147483647,7],"avail_no_missing":["4(0)","7(2)"],"object_location_counts":[{"shards":"4(0),7(2)","objects":3}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"3.17","version":"50'30","reported_seq":"252","reported_epoch":"192","state":"down","last_fresh":"2021-01-15T19:11:08.210556+0000","last_change":"2021-01-15T19:10:45.268830+0000","last_active":"2021-01-15T19:10:45.268471+0000","last_peered":"2021-01-15T19:10:44.298456+0000","last_clean":"2021-01-15T19:10:26.828058+0000","last_became_active":"2021-01-15T19:10:28.212577+0000","last_became_peered":"2021-01-15T19:10:28.212577+0000","last_unstale":"2021-01-15T19:11:08.210556+0000","last_undegraded":"2021-01-15T19:11:08.210556+0000","last_fullsized":"2021-01-15T19:11:08.210556+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"50'30","last_scrub_stamp":"2021-01-15T19:09:25.976466+0000","last_deep_scrub":"50'30","last_deep_scrub_stamp":"2021-01-15T19:09:04.507336+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.976466+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2147483647,2147483647],"acting":[7,2147483647,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"},{"start":"79","length":"1"}]},{"pgid":"1.14","version":"14'2","reported_seq":"137","reported_epoch":"192","state":"active+undersized","last_fresh":"2021-01-15T19:11:08.209559+0000","last_change":"2021-01-15T19:10:46.270534+0000","last_active":"2021-01-15T19:11:08.209559+0000","last_peered":"2021-01-15T19:11:08.209559+0000","last_clean":"2021-01-15T19:10:23.297118+0000","last_became_active":"2021-01-15T19:10:46.270534+0000","last_became_peered":"2021-01-15T19:10:46.270534+0000","last_unstale":"2021-01-15T19:11:08.209559+0000","last_undegraded":"2021-01-15T19:11:08.209559+0000","last_fullsized":"2021-01-15T19:10:46.268081+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"178'136","reported_seq":"551","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.345612+0000","last_change":"2021-01-15T19:10:46.778080+0000","last_active":"2021-01-15T19:11:10.345612+0000","last_peered":"2021-01-15T19:11:10.345612+0000","last_clean":"2021-01-15T19:11:10.345612+0000","last_became_active":"2021-01-15T19:09:15.343356+0000","last_became_peered":"2021-01-15T19:09:15.343356+0000","last_unstale":"2021-01-15T19:11:10.345612+0000","last_undegraded":"2021-01-15T19:11:10.345612+0000","last_fullsized":"2021-01-15T19:11:10.345612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":5,"last_scrub":"168'123","last_scrub_stamp":"2021-01-15T19:10:37.868333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:37.868333+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7331840,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":15,"num_read_kb":16614,"num_write":19,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2621440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"3.15","version":"174'88","reported_seq":"457","reported_epoch":"192","state":"down","last_fresh":"2021-01-15T19:11:08.209786+0000","last_change":"2021-01-15T19:10:45.262483+0000","last_active":"2021-01-15T19:10:45.262203+0000","last_peered":"2021-01-15T19:10:42.389278+0000","last_clean":"2021-01-15T19:10:26.827400+0000","last_became_active":"2021-01-15T19:10:28.207674+0000","last_became_peered":"2021-01-15T19:10:28.207674+0000","last_unstale":"2021-01-15T19:11:08.209786+0000","last_undegraded":"2021-01-15T19:11:08.209786+0000","last_fullsized":"2021-01-15T19:11:08.209786+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'32","last_scrub_stamp":"2021-01-15T19:09:03.911966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:03.911966+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9936896,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":2146,"num_write":51,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5455872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2147483647,3],"acting":[2147483647,2147483647,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"1"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"3.14","version":"165'108","reported_seq":"395","reported_epoch":"193","state":"down","last_fresh":"2021-01-15T19:11:10.345100+0000","last_change":"2021-01-15T19:10:45.264837+0000","last_active":"2021-01-15T19:10:45.264449+0000","last_peered":"2021-01-15T19:10:33.278808+0000","last_clean":"2021-01-15T19:10:26.284342+0000","last_became_active":"2021-01-15T19:10:28.212910+0000","last_became_peered":"2021-01-15T19:10:28.212910+0000","last_unstale":"2021-01-15T19:11:10.345100+0000","last_undegraded":"2021-01-15T19:11:10.345100+0000","last_fullsized":"2021-01-15T19:11:10.345100+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"148'94","last_scrub_stamp":"2021-01-15T19:10:16.897477+0000","last_deep_scrub":"148'94","last_deep_scrub_stamp":"2021-01-15T19:10:16.897477+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.897477+0000","log_size":108,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12705792,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":2353,"num_write":5,"num_write_kb":6008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,2147483647],"acting":[2147483647,2,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"1.11","version":"14'2","reported_seq":"142","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345921+0000","last_change":"2021-01-15T19:10:28.206594+0000","last_active":"2021-01-15T19:11:10.345921+0000","last_peered":"2021-01-15T19:11:10.345921+0000","last_clean":"2021-01-15T19:10:23.286754+0000","last_became_active":"2021-01-15T19:10:28.206594+0000","last_became_peered":"2021-01-15T19:10:28.206594+0000","last_unstale":"2021-01-15T19:11:10.345921+0000","last_undegraded":"2021-01-15T19:11:10.345921+0000","last_fullsized":"2021-01-15T19:10:28.203753+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:55.119702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:55.119702+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"404","reported_epoch":"192","state":"active+undersized","last_fresh":"2021-01-15T19:11:08.209711+0000","last_change":"2021-01-15T19:10:46.270198+0000","last_active":"2021-01-15T19:11:08.209711+0000","last_peered":"2021-01-15T19:11:08.209711+0000","last_clean":"2021-01-15T19:10:44.257599+0000","last_became_active":"2021-01-15T19:10:46.270198+0000","last_became_peered":"2021-01-15T19:10:46.270198+0000","last_unstale":"2021-01-15T19:11:08.209711+0000","last_undegraded":"2021-01-15T19:11:08.209711+0000","last_fullsized":"2021-01-15T19:10:46.265074+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:37.482819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_clean_scrub_stamp":"2021-01-15T19:10:37.482819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2147483647,2],"acting":[3,2147483647,2],"avail_no_missing":["3(0)","2(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"1.10","version":"14'2","reported_seq":"133","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345654+0000","last_change":"2021-01-15T19:10:46.267032+0000","last_active":"2021-01-15T19:11:10.345654+0000","last_peered":"2021-01-15T19:11:10.345654+0000","last_clean":"2021-01-15T19:09:16.341317+0000","last_became_active":"2021-01-15T19:10:46.267032+0000","last_became_peered":"2021-01-15T19:10:46.267032+0000","last_unstale":"2021-01-15T19:11:10.345654+0000","last_undegraded":"2021-01-15T19:11:10.345654+0000","last_fullsized":"2021-01-15T19:10:46.265140+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:16.218565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.218565+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"165'163","reported_seq":"603","reported_epoch":"192","state":"down","last_fresh":"2021-01-15T19:11:08.209441+0000","last_change":"2021-01-15T19:10:45.265637+0000","last_active":"2021-01-15T19:10:45.265316+0000","last_peered":"2021-01-15T19:10:33.270826+0000","last_clean":"2021-01-15T19:10:26.293959+0000","last_became_active":"2021-01-15T19:10:28.210672+0000","last_became_peered":"2021-01-15T19:10:28.210672+0000","last_unstale":"2021-01-15T19:11:08.209441+0000","last_undegraded":"2021-01-15T19:11:08.209441+0000","last_fullsized":"2021-01-15T19:11:08.209441+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"74'42","last_scrub_stamp":"2021-01-15T19:09:02.894120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.894120+0000","log_size":163,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15212544,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":25,"num_read_kb":32698,"num_write":43,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":57647104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2147483647,7],"acting":[2147483647,2147483647,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"1.4","version":"14'2","reported_seq":"159","reported_epoch":"192","state":"active+undersized","last_fresh":"2021-01-15T19:11:08.210328+0000","last_change":"2021-01-15T19:10:46.276115+0000","last_active":"2021-01-15T19:11:08.210328+0000","last_peered":"2021-01-15T19:11:08.210328+0000","last_clean":"2021-01-15T19:10:44.298014+0000","last_became_active":"2021-01-15T19:10:46.276115+0000","last_became_peered":"2021-01-15T19:10:46.276115+0000","last_unstale":"2021-01-15T19:11:08.210328+0000","last_undegraded":"2021-01-15T19:11:08.210328+0000","last_fullsized":"2021-01-15T19:10:46.269488+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"171'169","reported_seq":"597","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.347726+0000","last_change":"2021-01-15T19:10:28.208505+0000","last_active":"2021-01-15T19:11:10.347726+0000","last_peered":"2021-01-15T19:11:10.347726+0000","last_clean":"2021-01-15T19:10:26.810138+0000","last_became_active":"2021-01-15T19:10:28.208505+0000","last_became_peered":"2021-01-15T19:10:28.208505+0000","last_unstale":"2021-01-15T19:11:10.347726+0000","last_undegraded":"2021-01-15T19:10:28.205247+0000","last_fullsized":"2021-01-15T19:10:28.204970+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'62","last_scrub_stamp":"2021-01-15T19:09:14.332675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:14.332675+0000","log_size":169,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"stat_sum":{"num_bytes":14958592,"num_objects":11,"num_object_clones":9,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":17,"num_read_kb":18767,"num_write":59,"num_write_kb":12728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2147483647],"acting":[4,7,2147483647],"avail_no_missing":["4(0)","7(1)"],"object_location_counts":[{"shards":"4(0),7(1)","objects":11}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":"156","reported_epoch":"171","state":"stale+active+undersized","last_fresh":"2021-01-15T19:10:39.210985+0000","last_change":"2021-01-15T19:10:28.208168+0000","last_active":"2021-01-15T19:10:39.210985+0000","last_peered":"2021-01-15T19:10:39.210985+0000","last_clean":"2021-01-15T19:10:26.285035+0000","last_became_active":"2021-01-15T19:10:28.208168+0000","last_became_peered":"2021-01-15T19:10:28.208168+0000","last_unstale":"2021-01-15T19:10:45.259569+0000","last_undegraded":"2021-01-15T19:10:39.210985+0000","last_fullsized":"2021-01-15T19:10:28.204609+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.960545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.960545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"64'3","reported_seq":"196","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.344931+0000","last_change":"2021-01-15T19:10:38.897698+0000","last_active":"2021-01-15T19:11:10.344931+0000","last_peered":"2021-01-15T19:11:10.344931+0000","last_clean":"2021-01-15T19:11:10.344931+0000","last_became_active":"2021-01-15T19:10:23.277584+0000","last_became_peered":"2021-01-15T19:10:23.277584+0000","last_unstale":"2021-01-15T19:11:10.344931+0000","last_undegraded":"2021-01-15T19:11:10.344931+0000","last_fullsized":"2021-01-15T19:11:10.344931+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:10:38.897630+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:10:38.897630+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"206","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.347490+0000","last_change":"2021-01-15T19:10:46.279954+0000","last_active":"2021-01-15T19:11:10.347490+0000","last_peered":"2021-01-15T19:11:10.347490+0000","last_clean":"2021-01-15T19:10:44.261227+0000","last_became_active":"2021-01-15T19:10:46.279954+0000","last_became_peered":"2021-01-15T19:10:46.279954+0000","last_unstale":"2021-01-15T19:11:10.347490+0000","last_undegraded":"2021-01-15T19:11:10.347490+0000","last_fullsized":"2021-01-15T19:10:46.273505+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.103280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.103280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"167'88","reported_seq":"575","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.347690+0000","last_change":"2021-01-15T19:10:28.210615+0000","last_active":"2021-01-15T19:11:10.347690+0000","last_peered":"2021-01-15T19:11:10.347690+0000","last_clean":"2021-01-15T19:10:26.814405+0000","last_became_active":"2021-01-15T19:10:28.210615+0000","last_became_peered":"2021-01-15T19:10:28.210615+0000","last_unstale":"2021-01-15T19:11:10.347690+0000","last_undegraded":"2021-01-15T19:10:28.204203+0000","last_fullsized":"2021-01-15T19:10:28.203981+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'37","last_scrub_stamp":"2021-01-15T19:08:54.947352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:54.947352+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"stat_sum":{"num_bytes":3792896,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":36510,"num_write":17,"num_write_kb":3760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":8904704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2147483647],"acting":[4,7,2147483647],"avail_no_missing":["4(0)","7(1)"],"object_location_counts":[{"shards":"4(0),7(1)","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"1.6","version":"14'2","reported_seq":"144","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.209781+0000","last_change":"2021-01-15T19:10:51.158053+0000","last_active":"2021-01-15T19:11:08.209781+0000","last_peered":"2021-01-15T19:11:08.209781+0000","last_clean":"2021-01-15T19:11:08.209781+0000","last_became_active":"2021-01-15T19:10:23.284683+0000","last_became_peered":"2021-01-15T19:10:23.284683+0000","last_unstale":"2021-01-15T19:11:08.209781+0000","last_undegraded":"2021-01-15T19:11:08.209781+0000","last_fullsized":"2021-01-15T19:11:08.209781+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:51.157977+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:19.037889+0000","last_clean_scrub_stamp":"2021-01-15T19:10:51.157977+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"153'82","reported_seq":"567","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.344429+0000","last_change":"2021-01-15T19:10:28.209565+0000","last_active":"2021-01-15T19:11:10.344429+0000","last_peered":"2021-01-15T19:11:10.344429+0000","last_clean":"2021-01-15T19:10:26.287875+0000","last_became_active":"2021-01-15T19:10:28.209565+0000","last_became_peered":"2021-01-15T19:10:28.209565+0000","last_unstale":"2021-01-15T19:11:10.344429+0000","last_undegraded":"2021-01-15T19:10:28.201467+0000","last_fullsized":"2021-01-15T19:10:28.201249+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'69","last_scrub_stamp":"2021-01-15T19:10:01.942460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.942460+0000","log_size":82,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"stat_sum":{"num_bytes":622592,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":7,"num_read_kb":1825,"num_write":4,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,2147483647,4],"acting":[2,2147483647,4],"avail_no_missing":["2(0)","4(2)"],"object_location_counts":[{"shards":"2(0),4(2)","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"},{"start":"79","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"195","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.209811+0000","last_change":"2021-01-15T19:10:16.619367+0000","last_active":"2021-01-15T19:11:08.209811+0000","last_peered":"2021-01-15T19:11:08.209811+0000","last_clean":"2021-01-15T19:11:08.209811+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:11:08.209811+0000","last_undegraded":"2021-01-15T19:11:08.209811+0000","last_fullsized":"2021-01-15T19:11:08.209811+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:16.619291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.619291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"224","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.346056+0000","last_change":"2021-01-15T19:10:28.206487+0000","last_active":"2021-01-15T19:11:10.346056+0000","last_peered":"2021-01-15T19:11:10.346056+0000","last_clean":"2021-01-15T19:10:26.287284+0000","last_became_active":"2021-01-15T19:10:28.206487+0000","last_became_peered":"2021-01-15T19:10:28.206487+0000","last_unstale":"2021-01-15T19:11:10.346056+0000","last_undegraded":"2021-01-15T19:11:10.346056+0000","last_fullsized":"2021-01-15T19:10:28.204290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:28.194883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.194883+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"178'127","reported_seq":"611","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.209191+0000","last_change":"2021-01-15T19:10:46.781998+0000","last_active":"2021-01-15T19:11:08.209191+0000","last_peered":"2021-01-15T19:11:08.209191+0000","last_clean":"2021-01-15T19:11:08.209191+0000","last_became_active":"2021-01-15T19:09:51.207663+0000","last_became_peered":"2021-01-15T19:09:51.207663+0000","last_unstale":"2021-01-15T19:11:08.209191+0000","last_undegraded":"2021-01-15T19:11:08.209191+0000","last_fullsized":"2021-01-15T19:11:08.209191+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'114","last_scrub_stamp":"2021-01-15T19:10:28.056100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:28.056100+0000","log_size":127,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1581056,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":42,"num_write_kb":7408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":7340032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"216","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.209667+0000","last_change":"2021-01-15T19:09:53.454822+0000","last_active":"2021-01-15T19:11:08.209667+0000","last_peered":"2021-01-15T19:11:08.209667+0000","last_clean":"2021-01-15T19:11:08.209667+0000","last_became_active":"2021-01-15T19:09:48.201031+0000","last_became_peered":"2021-01-15T19:09:48.201031+0000","last_unstale":"2021-01-15T19:11:08.209667+0000","last_undegraded":"2021-01-15T19:11:08.209667+0000","last_fullsized":"2021-01-15T19:11:08.209667+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:53.454739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.454739+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"495","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.346402+0000","last_change":"2021-01-15T19:10:46.275951+0000","last_active":"2021-01-15T19:11:10.346402+0000","last_peered":"2021-01-15T19:11:10.346402+0000","last_clean":"2021-01-15T19:10:44.260749+0000","last_became_active":"2021-01-15T19:10:46.275951+0000","last_became_peered":"2021-01-15T19:10:46.275951+0000","last_unstale":"2021-01-15T19:11:10.346402+0000","last_undegraded":"2021-01-15T19:11:10.346402+0000","last_fullsized":"2021-01-15T19:10:46.271044+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'18","last_scrub_stamp":"2021-01-15T19:09:16.308952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.308952+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,2147483647],"acting":[4,2,2147483647],"avail_no_missing":["4(0)","2(1)"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"1.0","version":"14'2","reported_seq":"143","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345930+0000","last_change":"2021-01-15T19:10:28.206057+0000","last_active":"2021-01-15T19:11:10.345930+0000","last_peered":"2021-01-15T19:11:10.345930+0000","last_clean":"2021-01-15T19:09:16.341317+0000","last_became_active":"2021-01-15T19:10:28.206057+0000","last_became_peered":"2021-01-15T19:10:28.206057+0000","last_unstale":"2021-01-15T19:11:10.345930+0000","last_undegraded":"2021-01-15T19:11:10.345930+0000","last_fullsized":"2021-01-15T19:10:28.200499+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:16.218565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.218565+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"195","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.209720+0000","last_change":"2021-01-15T19:10:28.526252+0000","last_active":"2021-01-15T19:11:08.209720+0000","last_peered":"2021-01-15T19:11:08.209720+0000","last_clean":"2021-01-15T19:11:08.209720+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:11:08.209720+0000","last_undegraded":"2021-01-15T19:11:08.209720+0000","last_fullsized":"2021-01-15T19:11:08.209720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:28.526124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:28.526124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"27","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.346581+0000","last_change":"2021-01-15T19:10:46.279939+0000","last_active":"2021-01-15T19:11:10.346581+0000","last_peered":"2021-01-15T19:11:10.346581+0000","last_clean":"2021-01-15T19:08:47.197945+0000","last_became_active":"2021-01-15T19:10:46.279939+0000","last_became_peered":"2021-01-15T19:10:46.279939+0000","last_unstale":"2021-01-15T19:11:10.346581+0000","last_undegraded":"2021-01-15T19:11:10.346581+0000","last_fullsized":"2021-01-15T19:10:46.273358+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:42.893436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:42.893436+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"178'116","reported_seq":"588","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.345887+0000","last_change":"2021-01-15T19:10:46.784801+0000","last_active":"2021-01-15T19:11:10.345887+0000","last_peered":"2021-01-15T19:11:10.345887+0000","last_clean":"2021-01-15T19:11:10.345887+0000","last_became_active":"2021-01-15T19:09:21.202949+0000","last_became_peered":"2021-01-15T19:09:21.202949+0000","last_unstale":"2021-01-15T19:11:10.345887+0000","last_undegraded":"2021-01-15T19:11:10.345887+0000","last_fullsized":"2021-01-15T19:11:10.345887+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"174'106","last_scrub_stamp":"2021-01-15T19:10:43.182007+0000","last_deep_scrub":"174'106","last_deep_scrub_stamp":"2021-01-15T19:10:43.182007+0000","last_clean_scrub_stamp":"2021-01-15T19:10:43.182007+0000","log_size":116,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4374528,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":4708,"num_write":13,"num_write_kb":4272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"219","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.210204+0000","last_change":"2021-01-15T19:11:08.121869+0000","last_active":"2021-01-15T19:11:08.210204+0000","last_peered":"2021-01-15T19:11:08.210204+0000","last_clean":"2021-01-15T19:11:08.210204+0000","last_became_active":"2021-01-15T19:09:18.204537+0000","last_became_peered":"2021-01-15T19:09:18.204537+0000","last_unstale":"2021-01-15T19:11:08.210204+0000","last_undegraded":"2021-01-15T19:11:08.210204+0000","last_fullsized":"2021-01-15T19:11:08.210204+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_clean_scrub_stamp":"2021-01-15T19:10:05.030822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"146","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.346194+0000","last_change":"2021-01-15T19:10:28.206697+0000","last_active":"2021-01-15T19:11:10.346194+0000","last_peered":"2021-01-15T19:11:10.346194+0000","last_clean":"2021-01-15T19:10:26.287136+0000","last_became_active":"2021-01-15T19:10:28.206697+0000","last_became_peered":"2021-01-15T19:10:28.206697+0000","last_unstale":"2021-01-15T19:11:10.346194+0000","last_undegraded":"2021-01-15T19:11:10.346194+0000","last_fullsized":"2021-01-15T19:10:28.204202+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:55.119702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:55.119702+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"110","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.344965+0000","last_change":"2021-01-15T19:10:46.276794+0000","last_active":"2021-01-15T19:11:10.344965+0000","last_peered":"2021-01-15T19:11:10.344965+0000","last_clean":"2021-01-15T19:08:39.261385+0000","last_became_active":"2021-01-15T19:10:46.276794+0000","last_became_peered":"2021-01-15T19:10:46.276794+0000","last_unstale":"2021-01-15T19:11:10.344965+0000","last_undegraded":"2021-01-15T19:11:10.344965+0000","last_fullsized":"2021-01-15T19:10:46.265907+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:39.906692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:39.906692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,4],"acting":[2147483647,2,4],"avail_no_missing":["2(1)","4(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"111","reported_epoch":"192","state":"active+undersized","last_fresh":"2021-01-15T19:11:08.209167+0000","last_change":"2021-01-15T19:10:28.201710+0000","last_active":"2021-01-15T19:11:08.209167+0000","last_peered":"2021-01-15T19:11:08.209167+0000","last_clean":"2021-01-15T19:08:33.956428+0000","last_became_active":"2021-01-15T19:10:28.201710+0000","last_became_peered":"2021-01-15T19:10:28.201710+0000","last_unstale":"2021-01-15T19:11:08.209167+0000","last_undegraded":"2021-01-15T19:11:08.209167+0000","last_fullsized":"2021-01-15T19:10:28.200024+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.187515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.187515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"170","reported_epoch":"171","state":"stale+active+undersized","last_fresh":"2021-01-15T19:10:39.210822+0000","last_change":"2021-01-15T19:10:28.207566+0000","last_active":"2021-01-15T19:10:39.210822+0000","last_peered":"2021-01-15T19:10:39.210822+0000","last_clean":"2021-01-15T19:09:22.199740+0000","last_became_active":"2021-01-15T19:10:28.207566+0000","last_became_peered":"2021-01-15T19:10:28.207566+0000","last_unstale":"2021-01-15T19:10:45.259577+0000","last_undegraded":"2021-01-15T19:10:39.210822+0000","last_fullsized":"2021-01-15T19:10:28.204264+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"178'164","reported_seq":"685","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.344788+0000","last_change":"2021-01-15T19:10:46.778134+0000","last_active":"2021-01-15T19:11:10.344788+0000","last_peered":"2021-01-15T19:11:10.344788+0000","last_clean":"2021-01-15T19:11:10.344788+0000","last_became_active":"2021-01-15T19:09:51.202675+0000","last_became_peered":"2021-01-15T19:09:51.202675+0000","last_unstale":"2021-01-15T19:11:10.344788+0000","last_undegraded":"2021-01-15T19:11:10.344788+0000","last_fullsized":"2021-01-15T19:11:10.344788+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'90","last_scrub_stamp":"2021-01-15T19:10:02.902935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.902935+0000","log_size":164,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14041088,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":22,"num_read_kb":26000,"num_write":22,"num_write_kb":13712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12623872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"157","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.345208+0000","last_change":"2021-01-15T19:09:49.884000+0000","last_active":"2021-01-15T19:11:10.345208+0000","last_peered":"2021-01-15T19:11:10.345208+0000","last_clean":"2021-01-15T19:11:10.345208+0000","last_became_active":"2021-01-15T19:09:48.203471+0000","last_became_peered":"2021-01-15T19:09:48.203471+0000","last_unstale":"2021-01-15T19:11:10.345208+0000","last_undegraded":"2021-01-15T19:11:10.345208+0000","last_fullsized":"2021-01-15T19:11:10.345208+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:49.883930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:49.883930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.a","version":"14'2","reported_seq":"192","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.344609+0000","last_change":"2021-01-15T19:10:28.205674+0000","last_active":"2021-01-15T19:11:10.344609+0000","last_peered":"2021-01-15T19:11:10.344609+0000","last_clean":"2021-01-15T19:09:22.199641+0000","last_became_active":"2021-01-15T19:10:28.205674+0000","last_became_peered":"2021-01-15T19:10:28.205674+0000","last_unstale":"2021-01-15T19:11:10.344609+0000","last_undegraded":"2021-01-15T19:11:10.344609+0000","last_fullsized":"2021-01-15T19:10:28.199783+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_clean_scrub_stamp":"2021-01-15T19:10:07.176419+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"178'382","reported_seq":"836","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.345564+0000","last_change":"2021-01-15T19:10:47.292589+0000","last_active":"2021-01-15T19:11:10.345564+0000","last_peered":"2021-01-15T19:11:10.345564+0000","last_clean":"2021-01-15T19:11:10.345564+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:11:10.345564+0000","last_undegraded":"2021-01-15T19:11:10.345564+0000","last_fullsized":"2021-01-15T19:11:10.345564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'350","last_scrub_stamp":"2021-01-15T19:10:42.241708+0000","last_deep_scrub":"173'350","last_deep_scrub_stamp":"2021-01-15T19:10:42.241708+0000","last_clean_scrub_stamp":"2021-01-15T19:10:42.241708+0000","log_size":382,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25812992,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":33,"num_read_kb":92991,"num_write":87,"num_write_kb":20560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":"167","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345841+0000","last_change":"2021-01-15T19:10:46.280969+0000","last_active":"2021-01-15T19:11:10.345841+0000","last_peered":"2021-01-15T19:11:10.345841+0000","last_clean":"2021-01-15T19:10:44.260616+0000","last_became_active":"2021-01-15T19:10:46.280969+0000","last_became_peered":"2021-01-15T19:10:46.280969+0000","last_unstale":"2021-01-15T19:11:10.345841+0000","last_undegraded":"2021-01-15T19:11:10.345841+0000","last_fullsized":"2021-01-15T19:10:46.272483+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:25.339499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:25.339499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.b","version":"14'2","reported_seq":"237","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.345763+0000","last_change":"2021-01-15T19:10:31.174783+0000","last_active":"2021-01-15T19:11:10.345763+0000","last_peered":"2021-01-15T19:11:10.345763+0000","last_clean":"2021-01-15T19:11:10.345763+0000","last_became_active":"2021-01-15T19:10:28.212262+0000","last_became_peered":"2021-01-15T19:10:28.212262+0000","last_unstale":"2021-01-15T19:11:10.345763+0000","last_undegraded":"2021-01-15T19:11:10.345763+0000","last_fullsized":"2021-01-15T19:11:10.345763+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:31.174667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:31.174667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"178'273","reported_seq":"775","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.210017+0000","last_change":"2021-01-15T19:10:46.783556+0000","last_active":"2021-01-15T19:11:08.210017+0000","last_peered":"2021-01-15T19:11:08.210017+0000","last_clean":"2021-01-15T19:11:08.210017+0000","last_became_active":"2021-01-15T19:09:15.341093+0000","last_became_peered":"2021-01-15T19:09:15.341093+0000","last_unstale":"2021-01-15T19:11:08.210017+0000","last_undegraded":"2021-01-15T19:11:08.210017+0000","last_fullsized":"2021-01-15T19:11:08.210017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'253","last_scrub_stamp":"2021-01-15T19:10:39.050737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:39.050737+0000","log_size":273,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24305664,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":53,"num_read_kb":97876,"num_write":97,"num_write_kb":21032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"25","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345819+0000","last_change":"2021-01-15T19:10:46.269448+0000","last_active":"2021-01-15T19:11:10.345819+0000","last_peered":"2021-01-15T19:11:10.345819+0000","last_clean":"2021-01-15T19:08:35.214788+0000","last_became_active":"2021-01-15T19:10:46.269448+0000","last_became_peered":"2021-01-15T19:10:46.269448+0000","last_unstale":"2021-01-15T19:11:10.345819+0000","last_undegraded":"2021-01-15T19:11:10.345819+0000","last_fullsized":"2021-01-15T19:10:46.265441+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:28.840170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:10:28.840170+0000","last_clean_scrub_stamp":"2021-01-15T19:10:28.840170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"14'2","reported_seq":"109","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345713+0000","last_change":"2021-01-15T19:10:28.206502+0000","last_active":"2021-01-15T19:11:10.345713+0000","last_peered":"2021-01-15T19:11:10.345713+0000","last_clean":"2021-01-15T19:08:47.197231+0000","last_became_active":"2021-01-15T19:10:28.206502+0000","last_became_peered":"2021-01-15T19:10:28.206502+0000","last_unstale":"2021-01-15T19:11:10.345713+0000","last_undegraded":"2021-01-15T19:11:10.345713+0000","last_fullsized":"2021-01-15T19:10:28.203792+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:06.583774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:06.583774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"181'205","reported_seq":"645","reported_epoch":"192","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:08.209773+0000","last_change":"2021-01-15T19:10:28.205687+0000","last_active":"2021-01-15T19:11:08.209773+0000","last_peered":"2021-01-15T19:11:08.209773+0000","last_clean":"2021-01-15T19:10:26.740865+0000","last_became_active":"2021-01-15T19:10:28.205687+0000","last_became_peered":"2021-01-15T19:10:28.205687+0000","last_unstale":"2021-01-15T19:11:08.209773+0000","last_undegraded":"2021-01-15T19:10:28.199877+0000","last_fullsized":"2021-01-15T19:10:28.199644+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'120","last_scrub_stamp":"2021-01-15T19:09:30.324786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.324786+0000","log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"stat_sum":{"num_bytes":16359424,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":63,"num_read_kb":70678,"num_write":82,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2147483647,7],"acting":[3,2147483647,7],"avail_no_missing":["3(0)","7(2)"],"object_location_counts":[{"shards":"3(0),7(2)","objects":10}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.b","version":"0'0","reported_seq":"194","reported_epoch":"192","state":"active+clean","last_fresh":"2021-01-15T19:11:08.209854+0000","last_change":"2021-01-15T19:10:25.552359+0000","last_active":"2021-01-15T19:11:08.209854+0000","last_peered":"2021-01-15T19:11:08.209854+0000","last_clean":"2021-01-15T19:11:08.209854+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:11:08.209854+0000","last_undegraded":"2021-01-15T19:11:08.209854+0000","last_fullsized":"2021-01-15T19:11:08.209854+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:25.552278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:25.552278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"14'2","reported_seq":"109","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.346093+0000","last_change":"2021-01-15T19:10:28.207318+0000","last_active":"2021-01-15T19:11:10.346093+0000","last_peered":"2021-01-15T19:11:10.346093+0000","last_clean":"2021-01-15T19:08:47.197643+0000","last_became_active":"2021-01-15T19:10:28.207318+0000","last_became_peered":"2021-01-15T19:10:28.207318+0000","last_unstale":"2021-01-15T19:11:10.346093+0000","last_undegraded":"2021-01-15T19:11:10.346093+0000","last_fullsized":"2021-01-15T19:10:28.204163+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:18.403239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.403239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"174'105","reported_seq":"618","reported_epoch":"192","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:08.209761+0000","last_change":"2021-01-15T19:10:46.278531+0000","last_active":"2021-01-15T19:11:08.209761+0000","last_peered":"2021-01-15T19:11:08.209761+0000","last_clean":"2021-01-15T19:10:44.297773+0000","last_became_active":"2021-01-15T19:10:46.278531+0000","last_became_peered":"2021-01-15T19:10:46.278531+0000","last_unstale":"2021-01-15T19:11:08.209761+0000","last_undegraded":"2021-01-15T19:10:46.269078+0000","last_fullsized":"2021-01-15T19:10:46.268881+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'46","last_scrub_stamp":"2021-01-15T19:10:03.071240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:03.071240+0000","log_size":105,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":6905856,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":33974,"num_write":24,"num_write_kb":9304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2147483647],"acting":[7,4,2147483647],"avail_no_missing":["7(0)","4(1)"],"object_location_counts":[{"shards":"4(1),7(0)","objects":4}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":"146","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.344900+0000","last_change":"2021-01-15T19:10:46.269438+0000","last_active":"2021-01-15T19:11:10.344900+0000","last_peered":"2021-01-15T19:11:10.344900+0000","last_clean":"2021-01-15T19:09:16.350211+0000","last_became_active":"2021-01-15T19:10:46.269438+0000","last_became_peered":"2021-01-15T19:10:46.269438+0000","last_unstale":"2021-01-15T19:11:10.344900+0000","last_undegraded":"2021-01-15T19:11:10.344900+0000","last_fullsized":"2021-01-15T19:10:46.265638+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:01.966888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.966888+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"14'2","reported_seq":"42","reported_epoch":"193","state":"down","last_fresh":"2021-01-15T19:11:10.346079+0000","last_change":"2021-01-15T19:10:27.292093+0000","last_active":"2021-01-15T19:09:47.275830+0000","last_peered":"2021-01-15T19:09:46.273193+0000","last_clean":"2021-01-15T19:08:47.197265+0000","last_became_active":"2021-01-15T19:09:24.206432+0000","last_became_peered":"2021-01-15T19:09:24.206432+0000","last_unstale":"2021-01-15T19:11:10.346079+0000","last_undegraded":"2021-01-15T19:11:10.346079+0000","last_fullsized":"2021-01-15T19:11:10.346079+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:19.037889+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:19.037889+0000","last_clean_scrub_stamp":"2021-01-15T19:10:19.037889+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"181'402","reported_seq":"998","reported_epoch":"192","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:08.210167+0000","last_change":"2021-01-15T19:10:46.271726+0000","last_active":"2021-01-15T19:11:08.210167+0000","last_peered":"2021-01-15T19:11:08.210167+0000","last_clean":"2021-01-15T19:10:44.429005+0000","last_became_active":"2021-01-15T19:10:46.271726+0000","last_became_peered":"2021-01-15T19:10:46.271726+0000","last_unstale":"2021-01-15T19:11:08.210167+0000","last_undegraded":"2021-01-15T19:10:46.267718+0000","last_fullsized":"2021-01-15T19:10:46.267503+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'176","last_scrub_stamp":"2021-01-15T19:09:53.888744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.888744+0000","log_size":402,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":46006272,"num_objects":24,"num_object_clones":19,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":1,"num_read":101,"num_read_kb":150484,"num_write":178,"num_write_kb":48776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":87048192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,7,2],"acting":[2147483647,7,2],"avail_no_missing":["7(1)","2(2)"],"object_location_counts":[{"shards":"2(2),7(1)","objects":24}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.d","version":"0'0","reported_seq":"45","reported_epoch":"192","state":"active+undersized","last_fresh":"2021-01-15T19:11:08.209987+0000","last_change":"2021-01-15T19:10:28.202492+0000","last_active":"2021-01-15T19:11:08.209987+0000","last_peered":"2021-01-15T19:11:08.209987+0000","last_clean":"2021-01-15T19:08:35.218172+0000","last_became_active":"2021-01-15T19:10:28.202492+0000","last_became_peered":"2021-01-15T19:10:28.202492+0000","last_unstale":"2021-01-15T19:11:08.209987+0000","last_undegraded":"2021-01-15T19:11:08.209987+0000","last_fullsized":"2021-01-15T19:10:28.200709+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:25.416133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.416133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"14'2","reported_seq":"179","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.347679+0000","last_change":"2021-01-15T19:10:46.279376+0000","last_active":"2021-01-15T19:11:10.347679+0000","last_peered":"2021-01-15T19:11:10.347679+0000","last_clean":"2021-01-15T19:09:22.199740+0000","last_became_active":"2021-01-15T19:10:46.279376+0000","last_became_peered":"2021-01-15T19:10:46.279376+0000","last_unstale":"2021-01-15T19:11:10.347679+0000","last_undegraded":"2021-01-15T19:11:10.347679+0000","last_fullsized":"2021-01-15T19:10:46.273338+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"170'187","reported_seq":"622","reported_epoch":"193","state":"down","last_fresh":"2021-01-15T19:11:10.347446+0000","last_change":"2021-01-15T19:10:45.265689+0000","last_active":"2021-01-15T19:10:45.265340+0000","last_peered":"2021-01-15T19:10:38.227002+0000","last_clean":"2021-01-15T19:10:26.827342+0000","last_became_active":"2021-01-15T19:10:28.210680+0000","last_became_peered":"2021-01-15T19:10:28.210680+0000","last_unstale":"2021-01-15T19:11:10.347446+0000","last_undegraded":"2021-01-15T19:11:10.347446+0000","last_fullsized":"2021-01-15T19:11:10.347446+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'91","last_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_deep_scrub":"101'91","last_deep_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.016550+0000","log_size":187,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9297920,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":61,"num_read_kb":25912,"num_write":91,"num_write_kb":11152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,4,2147483647],"acting":[2147483647,4,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"1"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"93","reported_epoch":"192","state":"active+undersized","last_fresh":"2021-01-15T19:11:08.209737+0000","last_change":"2021-01-15T19:10:46.276087+0000","last_active":"2021-01-15T19:11:08.209737+0000","last_peered":"2021-01-15T19:11:08.209737+0000","last_clean":"2021-01-15T19:08:33.194477+0000","last_became_active":"2021-01-15T19:10:46.276087+0000","last_became_peered":"2021-01-15T19:10:46.276087+0000","last_unstale":"2021-01-15T19:11:08.209737+0000","last_undegraded":"2021-01-15T19:11:08.209737+0000","last_fullsized":"2021-01-15T19:10:46.268304+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:02.931607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.931607+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"14'2","reported_seq":"41","reported_epoch":"193","state":"down","last_fresh":"2021-01-15T19:11:10.345635+0000","last_change":"2021-01-15T19:10:45.263158+0000","last_active":"2021-01-15T19:10:26.286189+0000","last_peered":"2021-01-15T19:08:50.186756+0000","last_clean":"2021-01-15T19:08:47.198005+0000","last_became_active":"2021-01-15T19:08:50.186352+0000","last_became_peered":"2021-01-15T19:08:50.186352+0000","last_unstale":"2021-01-15T19:11:10.345635+0000","last_undegraded":"2021-01-15T19:11:10.345635+0000","last_fullsized":"2021-01-15T19:11:10.345635+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"176'150","reported_seq":"635","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.345681+0000","last_change":"2021-01-15T19:10:46.271371+0000","last_active":"2021-01-15T19:11:10.345681+0000","last_peered":"2021-01-15T19:11:10.345681+0000","last_clean":"2021-01-15T19:10:44.305735+0000","last_became_active":"2021-01-15T19:10:46.271371+0000","last_became_peered":"2021-01-15T19:10:46.271371+0000","last_unstale":"2021-01-15T19:11:10.345681+0000","last_undegraded":"2021-01-15T19:10:46.266351+0000","last_fullsized":"2021-01-15T19:10:46.266140+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'59","last_scrub_stamp":"2021-01-15T19:09:28.771226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.771226+0000","log_size":150,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":9117696,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":5385,"num_write":71,"num_write_kb":9088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,2147483647,7],"acting":[2,2147483647,7],"avail_no_missing":["2(0)","7(2)"],"object_location_counts":[{"shards":"2(0),7(2)","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.f","version":"0'0","reported_seq":"185","reported_epoch":"171","state":"stale+active+undersized","last_fresh":"2021-01-15T19:10:39.210968+0000","last_change":"2021-01-15T19:10:28.207876+0000","last_active":"2021-01-15T19:10:39.210968+0000","last_peered":"2021-01-15T19:10:39.210968+0000","last_clean":"2021-01-15T19:10:26.284995+0000","last_became_active":"2021-01-15T19:10:28.207876+0000","last_became_peered":"2021-01-15T19:10:28.207876+0000","last_unstale":"2021-01-15T19:10:45.259593+0000","last_undegraded":"2021-01-15T19:10:39.210968+0000","last_fullsized":"2021-01-15T19:10:28.204459+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"64'3","reported_seq":"216","reported_epoch":"193","state":"active+clean+remapped","last_fresh":"2021-01-15T19:11:10.345245+0000","last_change":"2021-01-15T19:10:40.986858+0000","last_active":"2021-01-15T19:11:10.345245+0000","last_peered":"2021-01-15T19:11:10.345245+0000","last_clean":"2021-01-15T19:11:10.345245+0000","last_became_active":"2021-01-15T19:10:29.205034+0000","last_became_peered":"2021-01-15T19:10:29.205034+0000","last_unstale":"2021-01-15T19:11:10.345245+0000","last_undegraded":"2021-01-15T19:11:10.345245+0000","last_fullsized":"2021-01-15T19:11:10.345245+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:10:40.986734+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:10:40.986734+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[{"shards":"2,4","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"172'193","reported_seq":"617","reported_epoch":"193","state":"down","last_fresh":"2021-01-15T19:11:10.347850+0000","last_change":"2021-01-15T19:10:45.269484+0000","last_active":"2021-01-15T19:10:45.265477+0000","last_peered":"2021-01-15T19:10:40.370173+0000","last_clean":"2021-01-15T19:10:27.290581+0000","last_became_active":"2021-01-15T19:10:28.209398+0000","last_became_peered":"2021-01-15T19:10:28.209398+0000","last_unstale":"2021-01-15T19:11:10.347850+0000","last_undegraded":"2021-01-15T19:11:10.347850+0000","last_fullsized":"2021-01-15T19:11:10.347850+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'93","last_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_deep_scrub":"100'93","last_deep_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.069708+0000","log_size":193,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12697600,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":22264,"num_write":89,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2147483647,4],"acting":[2147483647,2147483647,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.e","version":"0'0","reported_seq":"216","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.347624+0000","last_change":"2021-01-15T19:10:04.241798+0000","last_active":"2021-01-15T19:11:10.347624+0000","last_peered":"2021-01-15T19:11:10.347624+0000","last_clean":"2021-01-15T19:11:10.347624+0000","last_became_active":"2021-01-15T19:09:18.205134+0000","last_became_peered":"2021-01-15T19:09:18.205134+0000","last_unstale":"2021-01-15T19:11:10.347624+0000","last_undegraded":"2021-01-15T19:11:10.347624+0000","last_fullsized":"2021-01-15T19:11:10.347624+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:04.241731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:04.241731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.13","version":"14'2","reported_seq":"237","reported_epoch":"193","state":"active+clean","last_fresh":"2021-01-15T19:11:10.345645+0000","last_change":"2021-01-15T19:10:41.214323+0000","last_active":"2021-01-15T19:11:10.345645+0000","last_peered":"2021-01-15T19:11:10.345645+0000","last_clean":"2021-01-15T19:11:10.345645+0000","last_became_active":"2021-01-15T19:10:28.208230+0000","last_became_peered":"2021-01-15T19:10:28.208230+0000","last_unstale":"2021-01-15T19:11:10.345645+0000","last_undegraded":"2021-01-15T19:11:10.345645+0000","last_fullsized":"2021-01-15T19:11:10.345645+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:41.214187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:41.214187+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"178'119","reported_seq":"523","reported_epoch":"192","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:08.209325+0000","last_change":"2021-01-15T19:10:28.206472+0000","last_active":"2021-01-15T19:11:08.209325+0000","last_peered":"2021-01-15T19:11:08.209325+0000","last_clean":"2021-01-15T19:10:27.071633+0000","last_became_active":"2021-01-15T19:10:28.206472+0000","last_became_peered":"2021-01-15T19:10:28.206472+0000","last_unstale":"2021-01-15T19:11:08.209325+0000","last_undegraded":"2021-01-15T19:10:28.200211+0000","last_fullsized":"2021-01-15T19:10:28.200003+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":119,"ondisk_log_size":119,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"stat_sum":{"num_bytes":8413184,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":33,"num_read_kb":54352,"num_write":41,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,2147483647],"acting":[7,2,2147483647],"avail_no_missing":["7(0)","2(1)"],"object_location_counts":[{"shards":"2(1),7(0)","objects":7}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"},{"start":"79","length":"1"}]},{"pgid":"2.10","version":"0'0","reported_seq":"166","reported_epoch":"194","state":"active+clean","last_fresh":"2021-01-15T19:11:13.224905+0000","last_change":"2021-01-15T19:11:13.224980+0000","last_active":"2021-01-15T19:11:13.224905+0000","last_peered":"2021-01-15T19:11:13.224905+0000","last_clean":"2021-01-15T19:11:13.224905+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:11:13.224905+0000","last_undegraded":"2021-01-15T19:11:13.224905+0000","last_fullsized":"2021-01-15T19:11:13.224905+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:11:13.224905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:11:13.224905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"14'2","reported_seq":"27","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.346214+0000","last_change":"2021-01-15T19:10:46.279989+0000","last_active":"2021-01-15T19:11:10.346214+0000","last_peered":"2021-01-15T19:11:10.346214+0000","last_clean":"2021-01-15T19:08:47.197945+0000","last_became_active":"2021-01-15T19:10:46.279989+0000","last_became_peered":"2021-01-15T19:10:46.279989+0000","last_unstale":"2021-01-15T19:11:10.346214+0000","last_undegraded":"2021-01-15T19:10:46.273335+0000","last_fullsized":"2021-01-15T19:10:46.273175+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:16.945387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.945387+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"72","reported_epoch":"193","state":"active+undersized","last_fresh":"2021-01-15T19:11:10.345604+0000","last_change":"2021-01-15T19:10:46.280868+0000","last_active":"2021-01-15T19:11:10.345604+0000","last_peered":"2021-01-15T19:11:10.345604+0000","last_clean":"2021-01-15T19:09:10.329764+0000","last_became_active":"2021-01-15T19:10:46.280868+0000","last_became_peered":"2021-01-15T19:10:46.280868+0000","last_unstale":"2021-01-15T19:11:10.345604+0000","last_undegraded":"2021-01-15T19:11:10.345604+0000","last_fullsized":"2021-01-15T19:10:46.272131+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:37.898142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:37.898142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"181'206","reported_seq":"646","reported_epoch":"193","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:10.344305+0000","last_change":"2021-01-15T19:10:28.208975+0000","last_active":"2021-01-15T19:11:10.344305+0000","last_peered":"2021-01-15T19:11:10.344305+0000","last_clean":"2021-01-15T19:10:26.295820+0000","last_became_active":"2021-01-15T19:10:28.208975+0000","last_became_peered":"2021-01-15T19:10:28.208975+0000","last_unstale":"2021-01-15T19:11:10.344305+0000","last_undegraded":"2021-01-15T19:10:28.199928+0000","last_fullsized":"2021-01-15T19:10:28.199731+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"146'151","last_scrub_stamp":"2021-01-15T19:10:15.176740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:15.176740+0000","log_size":206,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"stat_sum":{"num_bytes":21102592,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":49,"num_read_kb":47272,"num_write":89,"num_write_kb":27312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":12124160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,4],"acting":[2147483647,2,4],"avail_no_missing":["2(1)","4(2)"],"object_location_counts":[{"shards":"2(1),4(2)","objects":15}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":276127744,"num_objects":183,"num_object_clones":134,"num_object_copies":549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":94,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":10,"num_read":643,"num_read_kb":761253,"num_write":1151,"num_write_kb":281256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":227622912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":441868288,"data_stored":773394432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3805,"internal_metadata":0},"log_size":3669,"ondisk_log_size":3669,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9184,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":25,"acting":26,"num_store_stats":8},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156888,"kb_used_data":102800,"kb_used_omap":0,"kb_used_meta":54080,"kb_avail":94214952,"statfs":{"total":96636764160,"available":96476110848,"internally_reserved":0,"allocated":105267200,"data_stored":199569850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":55377920},"hb_peers":[2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91400000000000003,"15min":0.92000000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1519999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.88600000000000001}]},{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97499999999999998,"15min":0.97099999999999997},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.341,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91400000000000003,"15min":0.91000000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.002,"15min":0.98999999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.748},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2050000000000001,"5min":1.276,"15min":1.276},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.379,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1399999999999999,"15min":1.157},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92400000000000004,"15min":0.96099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.087,"5min":1.427,"15min":1.427},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.1399999999999999,"15min":1.208},"min":{"1min":0.628,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4710000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.155,"15min":1.2010000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8180000000000001,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1040000000000001,"15min":1.139},"min":{"1min":0.71699999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.304,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Jan 15 19:11:02 2021","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.05},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.88800000000000001}]}]},{"osd":6,"up_from":0,"seq":566935683089,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84244,"kb_used_data":46476,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94287596,"statfs":{"total":96636764160,"available":96550498304,"internally_reserved":0,"allocated":47591424,"data_stored":87650503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2729999999999999,"5min":1.452,"15min":1.452},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0169999999999999,"15min":1.0469999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1819999999999999,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.169}]},{"osd":2,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.83599999999999997,"15min":0.85499999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.008,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.798},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.085,"15min":1.0920000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3480000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Jan 15 19:09:55 2021","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1479999999999999,"15min":1.1719999999999999},"min":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4910000000000001,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.494},{"interface":"front","average":{"1min":1.111,"5min":1.427,"15min":1.4790000000000001},"min":{"1min":0.70199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6519999999999999,"5min":10.584,"15min":10.584},"last":1.0580000000000001}]},{"osd":4,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.3620000000000001,"15min":1.3879999999999999},"min":{"1min":0.755,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1699999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.5469999999999999,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.018},{"interface":"front","average":{"1min":1.228,"5min":1.391,"15min":1.4179999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.673,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.3580000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.21},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.276}]},{"osd":7,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.488,"15min":1.512},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.7629999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.343},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.587,"15min":1.621},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0150000000000001,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.6499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94444,"kb_used_data":54436,"kb_used_omap":0,"kb_used_meta":40000,"kb_avail":94277396,"statfs":{"total":96636764160,"available":96540053504,"internally_reserved":0,"allocated":55742464,"data_stored":82736452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40960000},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.048,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94199999999999995,"15min":0.95299999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0549999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.181,"15min":1.198},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.506,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5369999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.95099999999999996}]},{"osd":5,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.681,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.141},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.3580000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.724,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.236},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.4219999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4350000000000001,"15min":1.4550000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.6819999999999999,"5min":1.96,"15min":1.96},"last":1.22},{"interface":"front","average":{"1min":1.367,"5min":1.5349999999999999,"15min":1.5629999999999999},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":1.847,"5min":2.024,"15min":2.024},"last":1.345}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135496,"kb_used_data":82816,"kb_used_omap":0,"kb_used_meta":52672,"kb_avail":94236344,"statfs":{"total":96636764160,"available":96498016256,"internally_reserved":0,"allocated":84803584,"data_stored":153480040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53936128},"hb_peers":[3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.54700000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2629999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95699999999999996,"15min":0.94899999999999995},"min":{"1min":0.76900000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.83099999999999996,"15min":0.81499999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0780000000000001,"15min":1.0860000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1220000000000001,"15min":1.1319999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4550000000000001,"5min":1.77,"15min":1.77},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.218,"15min":1.244},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7230000000000001,"5min":1.859,"15min":1.859},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.252,"15min":1.27},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6419999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1879999999999999,"15min":1.2050000000000001},"min":{"1min":0.83499999999999996,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.873}]},{"osd":6,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.353,"15min":1.3520000000000001},"min":{"1min":0.98399999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.8,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.393,"15min":1.395},"min":{"1min":1.069,"5min":1.014,"15min":1.014},"max":{"1min":1.952,"5min":2.133,"15min":2.133},"last":1.006}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32900,"kb_used_data":13628,"kb_used_omap":0,"kb_used_meta":19264,"kb_avail":94338940,"statfs":{"total":96636764160,"available":96603074560,"internally_reserved":0,"allocated":13955072,"data_stored":21795258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19726336},"hb_peers":[2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.755},{"interface":"front","average":{"1min":0.87,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0629999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.052,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":1.026},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.2,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.4370000000000003,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.151,"15min":1.1419999999999999},"min":{"1min":0.71199999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.258,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3080000000000001,"15min":1.3100000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.6030000000000002,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.462,"5min":1.4430000000000001,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.9900000000000002,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.1060000000000001}]},{"osd":7,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":0.93300000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":1.153},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.4219999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.254}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129564,"kb_used_data":77780,"kb_used_omap":0,"kb_used_meta":51776,"kb_avail":94242276,"statfs":{"total":96636764160,"available":96504090624,"internally_reserved":0,"allocated":79646720,"data_stored":154274664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53018624},"hb_peers":[2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3280000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.252,"5min":1.409,"15min":1.409},"last":1.276}]},{"osd":1,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91400000000000003,"15min":0.90600000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.617,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2050000000000001,"15min":1.1990000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.429}]},{"osd":2,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98099999999999998,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8300000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.377},"min":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3479999999999999,"5min":2.758,"15min":2.758},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.77100000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7170000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.88}]},{"osd":5,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.99299999999999999,"15min":1.0549999999999999},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3180000000000001,"5min":1.571,"15min":1.571},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.373,"5min":1.304,"15min":1.333},"min":{"1min":0.88900000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.637,"5min":2.637,"15min":2.637},"last":0.92700000000000005}]},{"osd":6,"last update":"Fri Jan 15 19:11:04 2021","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.476,"15min":1.5429999999999999},"min":{"1min":0.876,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.333,"15min":1.381},"min":{"1min":0.76900000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7310000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.95299999999999996}]}]},{"osd":5,"up_from":0,"seq":55834574892,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}]}} 2021-01-15T19:11:16.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pgp_num 21 2021-01-15T19:11:16.901 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:16.900+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:16.936 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:16.936 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:16.946 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:17.002 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:17.001+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:17.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:17.082 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:17.082 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:17.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:17.132 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:17.131+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:17.204 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:17.203+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:17.230 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:17.251 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:17.283 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:17.283 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:17.283 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:17.283 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:17.287 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:17.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:17.308 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:17.309 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:17.307+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:17.330 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:17.351 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:17.355 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:17.355 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 1 pgp_num to 21 2021-01-15T19:11:17.405 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:17.404+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:17.437 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:17.438 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:17.438 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:17.438 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:17.447 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:17.642 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:17.641+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:17.743 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:17.742+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:17.776 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:17.777 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:17.778 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 158.52533160900001, 2021-01-15T19:11:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:11:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:17.779 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:11:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.780 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:17.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:11:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:11:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:11:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.785 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:11:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:11:17.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:11:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:11:17.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:11:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:11:17.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:11:17.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:11:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:11:17.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:11:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:17.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:11:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:11:17.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:17.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:11:17.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:11:17.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:11:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:11:17.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:11:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:11:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:17.802 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 158.52514066500001, 2021-01-15T19:11:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:11:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:17.803 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:11:17.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.805 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:17.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:17.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:11:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:11:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:11:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:11:17.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:11:17.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:11:17.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:11:17.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:11:17.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:11:17.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:11:17.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:11:17.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:17.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:11:17.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:11:17.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:11:17.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:11:17.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.817 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:17.818 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:11:17.818 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:11:17.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.819 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:11:17.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:11:17.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:11:17.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:11:17.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:11:17.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:11:17.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.822 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:17.823 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:17.823 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.58618856800001, 2021-01-15T19:11:17.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:11:17.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:17.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.824 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:11:17.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.825 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:17.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:17.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:11:17.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:11:17.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:11:17.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:11:17.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:11:17.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:11:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:11:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:11:17.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:11:17.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:11:17.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:11:17.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:11:17.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:11:17.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:17.834 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:11:17.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:11:17.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:17.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:11:17.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:11:17.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:17.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:11:17.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:11:17.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:11:17.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:11:17.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.838 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.839 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:17.839 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:17.839 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.58346810699999, 2021-01-15T19:11:17.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:11:17.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:17.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:11:17.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:17.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:17.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:11:17.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:11:17.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:11:17.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:11:17.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:11:17.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:11:17.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:11:17.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:11:17.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:11:17.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:11:17.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:11:17.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:17.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:11:17.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:11:17.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:11:17.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:11:17.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:17.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:11:17.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:11:17.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:11:17.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:11:17.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:11:17.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:11:17.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:11:17.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:11:17.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.856 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:17.856 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:17.856 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.583355528, 2021-01-15T19:11:17.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:11:17.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:11:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.858 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:17.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:17.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:11:17.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:11:17.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:11:17.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:11:17.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:11:17.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:11:17.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:11:17.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:11:17.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:11:17.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:11:17.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:11:17.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:11:17.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:11:17.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:17.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:11:17.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:11:17.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:17.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:11:17.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:11:17.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:17.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:11:17.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:11:17.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:11:17.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:11:17.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.886 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.888 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:17.888 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:17.889 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.58322562399999, 2021-01-15T19:11:17.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:11:17.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:17.890 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.891 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:11:17.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.892 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:17.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:17.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:11:17.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:11:17.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:11:17.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:11:17.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:11:17.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:11:17.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:11:17.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:11:17.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:11:17.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:11:17.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:11:17.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:17.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:11:17.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:11:17.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:11:17.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:11:17.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:17.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:11:17.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:11:17.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:11:17.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:11:17.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:11:17.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:11:17.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:11:17.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:11:17.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:17.920 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:17.920 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.56002035500001, 2021-01-15T19:11:17.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:11:17.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:17.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.922 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:11:17.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.923 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:17.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:17.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:11:17.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:11:17.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:11:17.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:11:17.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:11:17.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:11:17.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:11:17.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:11:17.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:11:17.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.935 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.935 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:11:17.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:11:17.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:17.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:11:17.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:17.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:11:17.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:11:17.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:17.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:11:17.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:11:17.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:17.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:11:17.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:11:17.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:11:17.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:11:17.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.946 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.946 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.948 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:17.948 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:17.948 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.55990215700001, 2021-01-15T19:11:17.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:11:17.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:17.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.950 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.951 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:11:17.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.952 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:17.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:17.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:11:17.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:11:17.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:11:17.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:11:17.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:11:17.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:11:17.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:11:17.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:11:17.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:11:17.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:11:17.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:11:17.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:11:17.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:11:17.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:17.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:11:17.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:11:17.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:17.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:11:17.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:11:17.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:17.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:11:17.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:11:17.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:11:17.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:11:17.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.970 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.972 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:17.972 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:17.972 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.559789467, 2021-01-15T19:11:17.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:11:17.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:17.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:17.973 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:17.974 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:17.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:11:17.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.974 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:17.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:17.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:11:17.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:11:17.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:11:17.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:11:17.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:11:17.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:17.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:11:17.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:11:17.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:17.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:11:17.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:11:17.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:17.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:11:17.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:11:17.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:17.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:11:17.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:11:17.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:17.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:11:17.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:11:17.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:17.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:11:17.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:11:17.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:17.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:11:17.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:11:17.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:17.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:11:17.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:11:17.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.993 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:17.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:17.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.994 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:11:17.994 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:17.994 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.55418771699999, 2021-01-15T19:11:17.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:11:17.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:17.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:11:17.995 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:17.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:17.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:17.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:17.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:17.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:17.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:11:17.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:17.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:11:17.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:11:17.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:17.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:11:17.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:11:17.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:17.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:17.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:11:18.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:11:18.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:11:18.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:11:18.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:11:18.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:11:18.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:11:18.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:11:18.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.004 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:18.004 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:18.004 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.54571899300001, 2021-01-15T19:11:18.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:11:18.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:11:18.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.007 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:18.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:18.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:11:18.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:11:18.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:11:18.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:11:18.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:11:18.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:11:18.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:11:18.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:11:18.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:11:18.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:11:18.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:11:18.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:11:18.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:11:18.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:11:18.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:11:18.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.023 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.024 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:11:18.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:11:18.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:11:18.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:11:18.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:11:18.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:11:18.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.029 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.031 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:18.031 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:18.032 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.54120040999999, 2021-01-15T19:11:18.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:11:18.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.033 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.034 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.034 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:11:18.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.035 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:18.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:18.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:11:18.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:11:18.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:11:18.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:11:18.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:11:18.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:11:18.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:11:18.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:11:18.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:11:18.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:11:18.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:11:18.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:11:18.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:11:18.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:11:18.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:11:18.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:11:18.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:11:18.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:11:18.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:11:18.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:11:18.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:11:18.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.052 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.053 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:18.053 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:18.053 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 153.53618264100001, 2021-01-15T19:11:18.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:11:18.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:18.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.054 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:11:18.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.055 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:18.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:18.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:11:18.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:11:18.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:11:18.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:11:18.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:11:18.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:11:18.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:11:18.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:11:18.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:11:18.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:11:18.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:11:18.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:18.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:11:18.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:11:18.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:11:18.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:11:18.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:11:18.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:11:18.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:11:18.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:11:18.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.067 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.068 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:11:18.068 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:18.069 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.569058563, 2021-01-15T19:11:18.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:11:18.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.070 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:11:18.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.072 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:18.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:18.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:11:18.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:11:18.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:11:18.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:11:18.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:11:18.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:11:18.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:11:18.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:11:18.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:11:18.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:11:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:11:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:11:18.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:11:18.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:11:18.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:11:18.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:11:18.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:11:18.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:11:18.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:11:18.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:11:18.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:11:18.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:11:18.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:11:18.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:11:18.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:11:18.096 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.097 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.097 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:11:18.098 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:18.098 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 129.59187191500001, 2021-01-15T19:11:18.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:11:18.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.099 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:11:18.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.100 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:18.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:18.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:11:18.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:11:18.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:11:18.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:11:18.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:11:18.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:11:18.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:11:18.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:11:18.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:11:18.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:11:18.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:11:18.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:11:18.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:11:18.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:11:18.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:11:18.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:11:18.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:11:18.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:11:18.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:11:18.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:11:18.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:11:18.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:11:18.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:11:18.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:11:18.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:11:18.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.114 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.114 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.115 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:11:18.115 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:18.115 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.561324893999995, 2021-01-15T19:11:18.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:11:18.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:18.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.116 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.117 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:11:18.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.117 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:18.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:18.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:11:18.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:11:18.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:11:18.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.121 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.121 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:11:18.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:11:18.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.122 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.122 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:11:18.122 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:11:18.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:11:18.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:11:18.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:11:18.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:11:18.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:11:18.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:11:18.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:11:18.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:11:18.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:18.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:11:18.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:11:18.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:11:18.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:11:18.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:11:18.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:11:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:11:18.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:11:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.132 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:18.132 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:18.132 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 74.570666919000004, 2021-01-15T19:11:18.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:11:18.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.133 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:11:18.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.134 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:18.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:18.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:11:18.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.137 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:18.137 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:11:18.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.137 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.138 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:11:18.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:11:18.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.138 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:11:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:11:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.139 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:11:18.140 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:11:18.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:11:18.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:11:18.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:11:18.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:11:18.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:11:18.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:11:18.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:11:18.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:11:18.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:11:18.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:11:18.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:11:18.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:11:18.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:11:18.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:11:18.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:11:18.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:11:18.148 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.149 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:18.149 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:18.150 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 74.570527385999995, 2021-01-15T19:11:18.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:11:18.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.150 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.151 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:11:18.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.151 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.152 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:18.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:18.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:11:18.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.154 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:11:18.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:11:18.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.155 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.155 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:11:18.155 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:11:18.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.156 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.156 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.156 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:11:18.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:11:18.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.157 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.157 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:11:18.157 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:11:18.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:11:18.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:11:18.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:11:18.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:11:18.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:11:18.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:11:18.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:11:18.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:11:18.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:11:18.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:11:18.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:11:18.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:11:18.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:11:18.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:11:18.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:11:18.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:11:18.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.167 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:18.167 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:18.167 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 74.570412073, 2021-01-15T19:11:18.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:11:18.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.168 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:11:18.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.169 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:18.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.170 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.171 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:18.171 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:11:18.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.171 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.172 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:11:18.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:11:18.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.172 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.172 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.173 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:11:18.173 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:11:18.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.173 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:11:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:11:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:18.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:11:18.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:11:18.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:11:18.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:11:18.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:11:18.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:11:18.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:11:18.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:11:18.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:11:18.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:11:18.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:11:18.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:11:18.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:11:18.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:18.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:11:18.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:11:18.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:11:18.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:11:18.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.183 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.183 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.184 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:11:18.184 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:18.184 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 38.456940777, 2021-01-15T19:11:18.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:11:18.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:18.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:18.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:18.185 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:18.186 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:11:18.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.186 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:18.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:18.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:18.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:18.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.188 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:18.188 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:18.188 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:11:18.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.189 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:18.189 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:11:18.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:11:18.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.189 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.190 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:18.190 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:11:18.190 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:11:18.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:18.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:11:18.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:11:18.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:11:18.192 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:11:18.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:11:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:11:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:18.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:11:18.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:11:18.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:18.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:18.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:18.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:11:18.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:11:18.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:18.196 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:18.196 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:18.198 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:18.199 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:18.197+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:18.298 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:18.297+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:18.337 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:18.337 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:18.337 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:18.346 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:18.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:11:18.431 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:18.430+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:18.501 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:18.500+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:18.511 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:18.511 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:18.511 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:18.512 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:18.512 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:18.521 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:18.605 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:18.604+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:18.688 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:18.689 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:18.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.50222656, 2021-01-15T19:11:18.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:18.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:18.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:18.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:18.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:18.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:18.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.709 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.50206361900001, 2021-01-15T19:11:18.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.713 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.714 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:18.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:18.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:18.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.341020973, 2021-01-15T19:11:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:18.726 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.727 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:18.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:18.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:18.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:18.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:18.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:18.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:18.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:18.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:18.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.745 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.746 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.340913517, 2021-01-15T19:11:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.747 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.748 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:18.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:18.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:18.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:18.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:18.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:18.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:18.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:18.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:18.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:18.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:18.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:18.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:18.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:18.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:18.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:18.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:18.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:18.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:18.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:18.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:18.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:18.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:18.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:18.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:18.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:18.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:18.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:18.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:18.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:18.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:18.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.766 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.767 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:18.767 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:18.768 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 126.207846651, 2021-01-15T19:11:18.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:18.768 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.768 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.769 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:18.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.769 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:18.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:18.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:18.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:18.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:18.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:18.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:18.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.311878968, 2021-01-15T19:11:18.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:18.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:18.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:18.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:18.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:18.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.801 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.311748838, 2021-01-15T19:11:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.802 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.803 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:18.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:18.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:18.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:18.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 121.31166342900001, 2021-01-15T19:11:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:18.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:18.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:18.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:18.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:18.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.841 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:18.842 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.480489041, 2021-01-15T19:11:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.843 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.845 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:18.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:18.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:18.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:18.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:18.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:18.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:18.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:18.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.48040039200001, 2021-01-15T19:11:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:18.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:18.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:18.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:18.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:18.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:18.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:18.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:18.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:18.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:18.884 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:18.885 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 105.49689233399999, 2021-01-15T19:11:18.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:18.885 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.885 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.885 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.886 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:18.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.886 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:18.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:18.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:18.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:18.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:18.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:18.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:18.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:18.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:18.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:18.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:18.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:18.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:18.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:18.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:18.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:18.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:18.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:18.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:18.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:18.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:18.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:18.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:18.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:18.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:18.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:18.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:18.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:18.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.902 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:18.903 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 105.496760259, 2021-01-15T19:11:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.905 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:18.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.905 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:18.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:18.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:18.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:18.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:18.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:18.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:18.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:18.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:18.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:18.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:18.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:18.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:18.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:18.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:18.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:18.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:18.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:18.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:18.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:18.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:18.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:18.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:18.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:18.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:18.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:18.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:18.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:18.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:18.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:18.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:18.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.921 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.922 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:18.922 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:18.923 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.49362474199999, 2021-01-15T19:11:18.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:18.923 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.923 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:18.923 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.924 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.924 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:18.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.925 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:18.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:18.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:18.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:18.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:18.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:18.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:18.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:18.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:18.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:18.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:18.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:18.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:18.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:18.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:18.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:18.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:18.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:18.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:18.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:18.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:18.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:18.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:18.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:18.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:18.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:18.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:18.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:18.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:18.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:18.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.940 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.941 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:18.941 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:18.941 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.49352071600001, 2021-01-15T19:11:18.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:18.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:18.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:18.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:18.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:18.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:18.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:18.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:18.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:18.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:18.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:18.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:18.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:18.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:18.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:18.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:18.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:18.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:18.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:18.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:18.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:18.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:18.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:18.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:18.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:18.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:18.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:18.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:18.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:18.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:18.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:18.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:18.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:18.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.959 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.960 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:18.960 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:18.960 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.482533966999995, 2021-01-15T19:11:18.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:18.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:18.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:18.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:18.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:18.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:18.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:18.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:18.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:18.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:18.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:18.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:18.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:18.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:18.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:18.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:18.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:18.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:18.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:18.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:18.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:18.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:18.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:18.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:18.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.974 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.974 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.975 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:18.975 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:18.975 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.482438446000003, 2021-01-15T19:11:18.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:18.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.976 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.976 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.977 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:18.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.977 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:18.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:18.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:18.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:18.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:18.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:18.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:18.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:18.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:18.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:18.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:18.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:18.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:18.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:18.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:18.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:18.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:18.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:18.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:18.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:18.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:18.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:18.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:18.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:18.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.989 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:18.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:18.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.990 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:18.990 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:18.990 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.482327299999994, 2021-01-15T19:11:18.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:18.990 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:18.991 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:18.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:18.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:18.991 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:18.991 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:18.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.992 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:18.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:18.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:18.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:18.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:18.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:18.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:18.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:18.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:18.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:18.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:18.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:18.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:18.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:18.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:18.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:18.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:18.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:18.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:18.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:18.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:18.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:19.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:19.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:19.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:19.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:19.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:19.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:19.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:19.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:19.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.004 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.004 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.005 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:19.005 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:19.005 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.390552978000002, 2021-01-15T19:11:19.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:19.005 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.006 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.006 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.006 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:19.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:19.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:19.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:19.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:19.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:19.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:19.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:19.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:19.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:19.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:19.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:19.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:19.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:19.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:19.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:19.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:19.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:19.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:19.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:19.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:19.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:19.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:19.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.020 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:19.021 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:19.021 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 51.389269839000001, 2021-01-15T19:11:19.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:19.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.022 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:19.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:19.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:19.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:19.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:19.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:19.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:19.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:19.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:19.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:19.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:19.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:19.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:19.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:19.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:19.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:19.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:19.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:19.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:19.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:19.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:19.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:19.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:19.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.036 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.036 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:19.037 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:19.037 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.484141870999999, 2021-01-15T19:11:19.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:19.037 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.037 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.038 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.038 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.038 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.038 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:19.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.039 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:19.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:19.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:19.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:19.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:19.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:19.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:19.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:19.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:19.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.048 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.049 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.049 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:19.051 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:19.049+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:19.051 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:19.151 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:19.150+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:19.218 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:19.219 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:19.219 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:19.228 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:19.269 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:19.268+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:19.353 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:19.352+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:19.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:19.369 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:11:19.396 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:19.397 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:19.398 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:19.398 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:19.407 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:19.491 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:19.490+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:19.517 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:19.537 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:19.554 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:19.553+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:19.558 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:19.577 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:19.577 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:19.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:19.578 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:19.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.578 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:11:19.579 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:19.579 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 175.28764292400001, 2021-01-15T19:11:19.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:11:19.579 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.579 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.580 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.580 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.580 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.580 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:11:19.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.581 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:19.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:19.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:11:19.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:11:19.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:11:19.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:11:19.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:11:19.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:11:19.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:11:19.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:11:19.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:11:19.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:11:19.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:11:19.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:19.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:11:19.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:11:19.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:11:19.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:11:19.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:11:19.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:11:19.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:11:19.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:11:19.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:11:19.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:11:19.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:11:19.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:11:19.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:11:19.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:11:19.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.597 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:19.600 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:19.600 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.35588015799999, 2021-01-15T19:11:19.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:11:19.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.603 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:11:19.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.604 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:19.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:19.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:11:19.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:11:19.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:11:19.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:11:19.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:11:19.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:11:19.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:11:19.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:11:19.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:11:19.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:19.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:11:19.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:11:19.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:11:19.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:11:19.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:11:19.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:11:19.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:11:19.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:11:19.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:11:19.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:11:19.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:11:19.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:11:19.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:11:19.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:11:19.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.625 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.626 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:19.626 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:19.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.35359211700001, 2021-01-15T19:11:19.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:11:19.627 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.627 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.629 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.629 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:11:19.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.629 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:19.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:19.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:11:19.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:11:19.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:11:19.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:11:19.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:11:19.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:11:19.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:11:19.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:11:19.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:11:19.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:11:19.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:11:19.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:11:19.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:11:19.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:11:19.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:11:19.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:11:19.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:11:19.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:11:19.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:11:19.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:11:19.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:11:19.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.649 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.650 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:19.650 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:19.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.35100249199999, 2021-01-15T19:11:19.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:11:19.652 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.652 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.653 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:11:19.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.653 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:19.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:19.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:11:19.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:11:19.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:11:19.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:11:19.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:11:19.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:11:19.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:11:19.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:11:19.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:11:19.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:11:19.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:11:19.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:11:19.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:11:19.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:11:19.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:11:19.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:11:19.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:11:19.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:11:19.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:11:19.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:11:19.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:11:19.666 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.667 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.668 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:11:19.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:19.668 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.35009421399999, 2021-01-15T19:11:19.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:11:19.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.670 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:11:19.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.671 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:19.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:19.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:11:19.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:11:19.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:11:19.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:11:19.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:11:19.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:11:19.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:11:19.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:11:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:11:19.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:11:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:11:19.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:11:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:11:19.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:11:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:11:19.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:11:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:11:19.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:11:19.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:11:19.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 166.38141030700001, 2021-01-15T19:11:19.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:11:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:11:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:19.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:19.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:11:19.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:11:19.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:11:19.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:11:19.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:11:19.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:11:19.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:11:19.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:11:19.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:11:19.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:11:19.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:11:19.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:19.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:11:19.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:11:19.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:11:19.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:11:19.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:11:19.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:11:19.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:11:19.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:11:19.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:11:19.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:11:19.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:11:19.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:11:19.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:11:19.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:11:19.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.716 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.717 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:19.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:19.718 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 166.38126056999999, 2021-01-15T19:11:19.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:11:19.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.721 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:11:19.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.722 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:19.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:19.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:19.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:11:19.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:19.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:11:19.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:11:19.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:11:19.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:11:19.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:11:19.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:11:19.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:11:19.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:11:19.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:19.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:11:19.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:11:19.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:11:19.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:11:19.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:19.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:11:19.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:11:19.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:11:19.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:11:19.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:19.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:11:19.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:11:19.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:11:19.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:11:19.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:11:19.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:11:19.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:11:19.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:11:19.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.751 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:19.751 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:19.751 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 166.38110867200001, 2021-01-15T19:11:19.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:11:19.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.753 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:11:19.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:19.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:19.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:11:19.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:11:19.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:11:19.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:11:19.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:11:19.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:11:19.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:11:19.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:11:19.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:11:19.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:19.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:11:19.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:11:19.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:11:19.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:11:19.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:11:19.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:11:19.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:11:19.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:11:19.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:11:19.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:11:19.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:11:19.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:11:19.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:11:19.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:11:19.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.768 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.769 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:19.769 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:19.769 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 166.38098335999999, 2021-01-15T19:11:19.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:11:19.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.770 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.770 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:11:19.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:19.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:19.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:11:19.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:11:19.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:11:19.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:11:19.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:11:19.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:11:19.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:11:19.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:11:19.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:11:19.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:11:19.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:11:19.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:19.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:11:19.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:11:19.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:11:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:11:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:19.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:11:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:11:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:11:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:11:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:19.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:11:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:11:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:11:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:11:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:11:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:11:19.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:11:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:11:19.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.787 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:11:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.360150785, 2021-01-15T19:11:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:11:19.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.789 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:11:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:11:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:11:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:11:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:11:19.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:11:19.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:11:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:11:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:11:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:11:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:11:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:11:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:11:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:11:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:11:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:11:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:11:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:11:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:11:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:11:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:11:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:11:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.802 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:11:19.803 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.35385745299999, 2021-01-15T19:11:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:11:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:11:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:11:19.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:11:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:11:19.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:11:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:11:19.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:11:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:11:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:11:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:11:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:11:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:11:19.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:19.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:11:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:11:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:11:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:11:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:11:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:11:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:11:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:11:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:11:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:19.820 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.39179794, 2021-01-15T19:11:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:11:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:11:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:11:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:11:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:11:19.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:11:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:11:19.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:11:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:11:19.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:11:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:11:19.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:11:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:11:19.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:11:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:11:19.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:11:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:11:19.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:11:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:11:19.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:11:19.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:11:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:11:19.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:11:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:11:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:11:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:11:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:11:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.840 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.32942596000001, 2021-01-15T19:11:19.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:11:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.842 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:11:19.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:19.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:11:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:11:19.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:11:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:11:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:11:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:11:19.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:11:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:11:19.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:11:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:11:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:11:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:11:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:11:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:11:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:11:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:11:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:11:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:11:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:11:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:11:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:11:19.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.859 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.860 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.326123459, 2021-01-15T19:11:19.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:11:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.862 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.863 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:11:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:19.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:19.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:11:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:11:19.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:11:19.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:11:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:11:19.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:11:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:11:19.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:11:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:11:19.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:11:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:11:19.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:11:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:11:19.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:11:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:19.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:11:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:11:19.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:11:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:11:19.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:11:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.32480159299999, 2021-01-15T19:11:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:11:19.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.879 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:11:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:11:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:11:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:11:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:11:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:11:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:11:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:11:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:11:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:11:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:11:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:11:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:11:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:11:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:11:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:11:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:11:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:11:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:11:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.32443133999999, 2021-01-15T19:11:19.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:11:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:11:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:11:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:11:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:11:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:11:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:11:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:11:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:11:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:11:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:11:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:11:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:11:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:11:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:11:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:11:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:11:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:11:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:11:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:11:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:11:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:11:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:11:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.907 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:19.908 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.32409694699999, 2021-01-15T19:11:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:11:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:11:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.910 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:11:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:11:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:11:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:11:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:11:19.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:11:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:11:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:11:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:11:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:11:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:11:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:11:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:11:19.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:11:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:11:19.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:11:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:11:19.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:11:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:11:19.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.922 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:11:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 146.382835688, 2021-01-15T19:11:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:11:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.923 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:19.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:19.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:11:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:11:19.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:11:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:11:19.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:11:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:11:19.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:11:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:11:19.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:11:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:11:19.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:11:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:11:19.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:11:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:11:19.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:11:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:11:19.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:11:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:11:19.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:11:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:11:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.390472816, 2021-01-15T19:11:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:11:19.936 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.937 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:11:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:11:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:11:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:11:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:11:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:11:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:11:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:11:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:11:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:11:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:11:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:11:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:11:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:11:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:11:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:11:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:11:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:11:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:11:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:11:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:11:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:11:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:11:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:11:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:11:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:11:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:11:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:19.953 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.36047861200001, 2021-01-15T19:11:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:11:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:19.954 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:11:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:19.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:11:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:11:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:11:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:11:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:11:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:11:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:11:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:11:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:11:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:11:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:11:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:11:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:11:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:11:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:11:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:11:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:11:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:19.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:11:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:11:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:19.967 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:19.968 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:19.970 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:19.971 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:19.971 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:19.971 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:19.971 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:19.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:19.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:19.975 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:19.969+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:19.975 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:20.070 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:20.069+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:20.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:20.137 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:20.146 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:20.172 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:20.170+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:20.273 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:20.272+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:20.310 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:20.311 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:20.320 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:20.404 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:20.403+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:20.475 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:20.474+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:20.485 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:20.486 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:20.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:20.579 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:20.578+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:20.660 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:20.660 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:20.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:20.676 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:20.675+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:20.777 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:20.776+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:20.832 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:20.833 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:20.842 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:20.886 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:20.885+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:20.967 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:20.968 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:20.977 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:21.018 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:21.017+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:21.081 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:21.079+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:21.123 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:21.123 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:21.133 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:21.217 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:21.216+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:21.282 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:21.281+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:21.306 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:21.306 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:21.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:21.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:11:21.399 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:21.398+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:21.484 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:21.482+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:21.490 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:21.490 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:21.500 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:21.585 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:21.584+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:21.673 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:21.673 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:21.684 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:21.686 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:21.685+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:21.788 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:21.787+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:21.815 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:21.816 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:21.825 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:21.889 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:21.888+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:21.957 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:21.957 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:21.967 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:21.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:22.011 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:22.010+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:22.092 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:22.091+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:22.104 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:22.105 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:22.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:22.121 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:22.142 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:22.162 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:22.183 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:22.199 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:22.198+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:22.204 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:22.225 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:22.225 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:22.225 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:22.225 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:22.226 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:22.226 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:22.226 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:22.226 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:22.227 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:22.227 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:22.227 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:22.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:22.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:22.281 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:22.281 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:22.281 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:22.282 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:22.282 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:22.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:22.331 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:22.330+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:22.379 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:22.379 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:22.379 INFO:tasks.thrashosds.thrasher:inject_pause on 4 2021-01-15T19:11:22.380 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2021-01-15T19:11:22.380 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2021-01-15T19:11:22.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2021-01-15T19:11:22.381 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:11:22.425 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:22.424+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:22.465 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:22.465 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:22.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:22.466 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:22.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.466 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:22.466 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:22.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.277095156, 2021-01-15T19:11:22.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:22.467 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.467 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.468 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:22.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.469 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:22.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:22.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:22.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:22.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:22.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:22.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:22.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:22.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:22.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:22.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:22.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:22.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:22.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:22.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:22.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:22.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:22.487 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.487 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:22.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:22.490 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.27693221499999, 2021-01-15T19:11:22.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:22.491 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.493 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.493 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:22.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.494 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:22.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:22.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:22.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:22.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:22.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:22.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:22.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:22.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:22.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:22.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:22.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:22.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:22.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:22.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:22.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:22.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:22.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.514 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.516 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:22.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:22.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.11588956899999, 2021-01-15T19:11:22.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:22.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.520 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.520 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:22.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.521 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:22.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:22.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:22.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:22.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:22.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:22.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:22.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:22.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:22.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:22.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:22.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:22.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:22.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:22.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:22.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:22.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:22.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:22.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:22.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:22.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:22.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:22.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:22.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:22.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:22.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:22.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:22.535 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.536 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.536 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:22.537 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:22.537 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.11578211299999, 2021-01-15T19:11:22.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:22.537 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.537 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.538 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.538 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:22.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.539 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:22.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:22.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:22.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:22.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:22.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:22.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:22.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:22.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:22.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:22.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:22.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:22.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:22.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:22.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:22.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:22.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:22.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:22.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:22.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:22.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:22.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:22.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:22.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:22.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:22.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:22.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:22.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:22.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:22.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:22.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:22.554 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.554 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.555 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.555 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:22.555 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:22.556 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 129.98271524699999, 2021-01-15T19:11:22.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:22.556 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.556 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.556 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.557 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.557 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:22.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.557 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:22.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:22.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:22.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:22.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:22.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:22.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:22.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:22.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:22.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:22.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:22.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:22.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:22.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:22.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:22.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:22.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:22.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:22.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:22.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:22.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:22.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:22.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:22.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:22.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:22.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:22.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:22.572 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.572 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.572 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.573 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:22.573 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:22.573 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.08674756400001, 2021-01-15T19:11:22.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:22.574 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.574 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:22.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.574 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.575 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:22.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.575 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:22.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:22.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:22.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:22.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:22.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:22.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:22.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:22.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:22.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:22.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:22.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:22.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:22.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:22.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:22.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:22.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:22.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:22.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:22.586 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.586 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.586 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.587 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:22.587 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:22.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.086617434, 2021-01-15T19:11:22.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:22.587 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.588 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:22.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.589 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:22.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.589 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:22.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:22.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:22.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:22.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:22.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:22.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:22.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:22.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:22.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:22.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:22.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:22.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:22.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:22.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:22.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:22.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:22.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:22.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:22.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.086532025, 2021-01-15T19:11:22.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:22.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:22.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.602 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:22.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.603 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:22.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:22.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:22.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:22.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:22.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:22.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:22.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:22.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:22.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:22.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:22.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:22.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:22.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:22.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:22.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:22.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:22.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:22.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:22.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:22.614 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:22.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.255357637, 2021-01-15T19:11:22.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:22.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:22.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:22.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:22.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:22.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:22.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:22.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:22.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:22.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:22.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:22.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:22.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:22.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:22.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:22.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:22.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:22.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:22.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:22.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:22.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:22.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:22.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:22.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:22.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:22.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:22.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:22.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:22.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:22.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:22.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:22.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:22.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:22.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:22.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.636 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.638 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:22.638 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:22.638 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.255268988, 2021-01-15T19:11:22.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:22.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.640 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:22.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.641 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:22.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:22.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:22.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:22.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:22.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:22.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:22.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:22.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:22.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:22.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:22.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:22.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:22.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:22.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:22.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:22.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:22.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:22.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:22.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:22.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:22.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:22.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:22.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:22.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:22.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:22.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:22.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:22.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:22.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:22.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:22.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:22.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:22.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.665 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.666 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:22.667 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:22.667 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.27176093, 2021-01-15T19:11:22.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:22.668 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.668 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.668 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.669 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:22.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:22.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:22.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:22.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:22.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:22.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:22.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:22.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:22.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:22.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:22.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:22.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:22.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:22.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:22.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:22.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:22.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:22.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:22.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:22.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:22.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:22.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:22.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:22.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:22.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:22.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:22.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:22.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:22.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:22.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:22.696 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 109.271628855, 2021-01-15T19:11:22.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:22.696 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.697 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.698 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:22.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.699 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:22.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:22.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:22.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:22.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:22.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:22.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:22.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:22.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:22.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:22.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:22.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:22.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:22.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:22.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:22.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:22.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:22.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:22.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:22.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:22.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:22.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:22.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:22.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:22.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:22.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:22.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:22.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:22.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:22.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:22.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:22.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:22.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.721 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:22.722 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:22.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 104.268493338, 2021-01-15T19:11:22.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:22.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.725 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:22.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.725 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:22.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:22.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:22.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:22.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:22.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:22.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:22.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:22.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:22.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:22.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:22.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:22.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:22.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:22.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:22.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:22.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:22.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:22.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:22.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:22.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:22.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:22.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:22.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:22.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:22.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:22.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:22.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:22.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.745 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:22.745 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:22.745 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 104.268389312, 2021-01-15T19:11:22.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:22.746 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.746 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.747 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:22.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.748 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:22.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:22.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:22.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:22.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:22.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:22.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:22.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:22.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:22.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:22.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:22.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:22.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:22.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:22.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:22.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:22.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:22.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:22.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:22.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:22.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:22.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:22.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:22.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:22.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:22.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:22.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:22.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:22.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:22.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:22.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:22.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:22.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.771 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.771 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:22.772 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:22.772 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.257402562999999, 2021-01-15T19:11:22.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:22.773 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.773 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.774 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.774 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.774 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:22.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.775 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:22.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:22.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:22.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:22.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:22.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:22.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:22.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:22.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:22.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:22.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:22.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:22.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:22.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:22.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:22.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:22.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:22.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:22.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:22.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:22.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:22.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:22.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.792 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:22.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:22.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.257307041999994, 2021-01-15T19:11:22.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:22.795 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.795 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.796 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.796 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.796 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:22.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:22.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:22.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:22.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:22.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:22.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:22.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:22.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:22.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:22.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:22.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:22.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:22.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:22.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:22.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:22.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:22.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:22.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:22.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:22.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:22.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:22.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:22.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.815 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:22.817 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:22.817 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 79.257195895999999, 2021-01-15T19:11:22.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:22.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.818 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.819 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:22.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.819 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:22.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:22.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:22.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:22.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:22.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:22.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:22.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:22.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:22.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:22.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:22.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:22.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:22.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:22.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:22.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:22.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:22.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:22.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:22.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:22.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:22.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:22.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.831 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.832 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:22.832 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:22.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.165421574, 2021-01-15T19:11:22.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:22.832 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.833 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.834 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:22.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.834 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:22.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:22.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:22.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:22.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:22.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:22.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:22.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:22.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:22.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:22.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:22.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:22.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:22.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:22.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:22.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:22.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:22.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:22.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:22.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:22.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:22.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:22.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:22.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.848 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:22.848 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:22.848 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.164138434999998, 2021-01-15T19:11:22.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:22.848 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.849 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:22.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.850 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:22.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:22.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:22.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:22.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:22.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:22.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:22.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:22.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:22.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:22.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:22.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:22.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:22.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:22.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:22.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:22.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:22.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:22.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:22.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:22.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:22.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:22.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:22.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:22.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:22.862 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.863 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:22.864 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:22.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.259010467000003, 2021-01-15T19:11:22.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:22.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:22.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:22.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:22.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:22.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:22.865 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:22.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.866 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:22.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:22.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:22.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:22.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:22.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:22.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:22.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:22.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:22.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:22.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:22.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:22.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:22.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:22.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:22.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:22.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:22.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:22.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:22.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:22.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:22.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:22.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:22.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:22.875 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.875 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.875 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.876 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:22.876 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:22.876 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:22.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:22.878 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:22.879 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2021-01-15T19:11:22.879 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:22.880 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:22.878+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:22.981 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:22.979+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:23.006 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:23.006 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:23.006 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:23.007 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:23.015 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:23.109 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:23.108+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:23.182 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:23.181+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:23.184 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:23.185 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:23.185 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:23.186 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:23.186 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:23.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:23.289 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:23.288+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:23.367 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:23.368 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:23.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:23.368 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:23.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.369 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:11:23.369 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:23.369 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 179.07917850699999, 2021-01-15T19:11:23.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:11:23.370 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.370 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.370 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.370 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.371 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:11:23.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.371 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:23.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:23.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:11:23.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:11:23.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:11:23.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:11:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:11:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:11:23.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:11:23.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:11:23.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:11:23.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:11:23.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:11:23.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:23.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:11:23.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:11:23.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:11:23.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:11:23.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:11:23.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:11:23.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:11:23.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:11:23.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:11:23.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:11:23.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:11:23.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:11:23.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:11:23.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:11:23.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.388 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:23.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:23.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 174.147415741, 2021-01-15T19:11:23.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:11:23.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.390 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.390 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:11:23.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.391 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:23.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:23.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:11:23.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:11:23.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:11:23.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:11:23.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:11:23.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:11:23.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:11:23.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:11:23.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:11:23.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:11:23.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:11:23.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:23.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:11:23.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:11:23.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:11:23.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:11:23.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:11:23.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:11:23.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:11:23.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:11:23.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:11:23.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:11:23.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:11:23.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:11:23.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:11:23.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:11:23.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.406 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.407 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:23.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:23.407 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 174.14512769999999, 2021-01-15T19:11:23.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:11:23.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.409 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.409 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:11:23.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.409 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:23.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:23.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:11:23.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:11:23.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:11:23.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:11:23.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:11:23.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:11:23.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:11:23.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:11:23.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:11:23.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:11:23.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:11:23.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:11:23.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:11:23.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:11:23.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:11:23.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:11:23.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:11:23.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:11:23.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:11:23.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:11:23.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:11:23.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.422 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.422 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.422 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:23.423 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:23.423 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 174.142538075, 2021-01-15T19:11:23.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:11:23.423 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.423 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.424 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.424 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:11:23.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.425 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:23.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:23.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:11:23.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:11:23.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:11:23.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:11:23.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:11:23.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:11:23.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:11:23.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:11:23.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:11:23.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:11:23.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:11:23.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:11:23.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:11:23.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:11:23.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:11:23.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:11:23.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:11:23.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:11:23.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:11:23.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:11:23.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:11:23.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.438 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:11:23.438 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:23.438 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 174.14162979700001, 2021-01-15T19:11:23.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:11:23.439 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.439 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.439 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.440 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:11:23.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.440 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:23.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:23.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:11:23.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:11:23.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:11:23.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:11:23.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:11:23.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:11:23.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:11:23.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:11:23.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:11:23.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:11:23.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:11:23.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:23.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:11:23.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:11:23.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:11:23.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:11:23.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:11:23.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:11:23.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:11:23.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:11:23.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.452 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.453 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:23.453 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:23.453 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.17294588999999, 2021-01-15T19:11:23.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:11:23.453 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.454 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.454 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.454 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:11:23.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:23.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:11:23.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:11:23.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:11:23.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:11:23.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:11:23.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:11:23.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:11:23.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:11:23.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:11:23.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:11:23.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:23.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:11:23.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:11:23.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:11:23.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:11:23.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:11:23.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:11:23.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:11:23.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:11:23.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:11:23.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:11:23.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:11:23.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:11:23.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:11:23.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:11:23.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.469 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.469 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.470 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:23.470 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:23.470 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.17279615300001, 2021-01-15T19:11:23.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:11:23.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.472 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:11:23.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:23.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:23.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:23.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:11:23.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:23.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:11:23.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:11:23.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:11:23.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:11:23.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:11:23.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:11:23.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:11:23.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:11:23.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:23.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:11:23.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:11:23.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:11:23.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:11:23.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:23.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:11:23.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:11:23.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:11:23.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:11:23.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:23.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:11:23.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:11:23.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:11:23.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:11:23.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:11:23.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:11:23.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:11:23.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:11:23.487 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.488 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.488 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.488 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:23.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:23.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.17264425499999, 2021-01-15T19:11:23.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:11:23.489 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.489 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.490 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.490 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:11:23.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:23.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:23.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:11:23.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:11:23.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:11:23.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:11:23.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:11:23.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:11:23.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:11:23.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:11:23.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:11:23.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:11:23.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:11:23.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:23.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:11:23.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:11:23.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:11:23.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:11:23.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:11:23.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:11:23.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:11:23.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:11:23.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:11:23.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:11:23.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:11:23.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:11:23.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:11:23.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:11:23.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.505 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.506 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:23.506 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:23.506 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 170.172518943, 2021-01-15T19:11:23.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:11:23.507 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.507 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.508 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:11:23.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.508 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:23.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:23.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:11:23.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:11:23.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:11:23.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:11:23.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:11:23.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:11:23.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:11:23.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:11:23.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:11:23.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:23.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:11:23.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:11:23.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:11:23.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:11:23.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:23.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:11:23.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:11:23.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:11:23.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:11:23.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:23.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:11:23.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:11:23.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:11:23.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:11:23.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:11:23.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:11:23.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:11:23.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:11:23.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.524 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:11:23.524 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:23.524 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.15168636799999, 2021-01-15T19:11:23.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:11:23.525 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.525 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.525 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.526 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.526 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:11:23.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.526 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:23.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:23.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:11:23.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:11:23.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:11:23.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:11:23.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:11:23.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:11:23.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:11:23.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:11:23.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:11:23.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:11:23.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:11:23.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:11:23.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:11:23.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:11:23.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:11:23.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:11:23.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:11:23.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:11:23.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:11:23.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:11:23.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:11:23.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.538 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.539 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.539 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:11:23.539 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:23.540 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.145393036, 2021-01-15T19:11:23.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:11:23.540 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.540 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.540 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.541 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.541 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.541 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:11:23.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.541 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:23.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:23.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:11:23.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:11:23.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:11:23.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:11:23.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:11:23.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:11:23.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:11:23.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:11:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:11:23.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:11:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:11:23.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:23.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:11:23.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:11:23.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:11:23.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:11:23.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:11:23.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:11:23.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:11:23.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:11:23.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.553 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.553 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:11:23.554 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:23.554 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.18333352299999, 2021-01-15T19:11:23.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:11:23.554 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.554 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.555 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.555 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:11:23.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.556 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:23.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:23.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:11:23.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:11:23.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:11:23.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:11:23.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:11:23.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:11:23.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:11:23.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:11:23.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:11:23.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:11:23.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:11:23.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:23.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:11:23.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:11:23.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:11:23.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:11:23.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:11:23.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:11:23.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:11:23.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:11:23.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:11:23.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:11:23.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:11:23.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:11:23.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:11:23.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:11:23.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.570 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.571 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:23.571 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:23.571 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.12096154299999, 2021-01-15T19:11:23.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:11:23.572 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.572 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.572 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.573 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:11:23.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.573 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:23.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:23.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:11:23.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:11:23.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:11:23.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:11:23.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:11:23.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:11:23.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:11:23.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:11:23.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:11:23.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:11:23.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:11:23.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:11:23.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:11:23.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:11:23.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:11:23.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:11:23.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:11:23.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:11:23.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:11:23.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:11:23.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:11:23.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.585 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.586 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:23.586 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:23.586 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.11765904200001, 2021-01-15T19:11:23.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:11:23.587 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.587 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.588 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:11:23.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:23.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:23.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:11:23.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:11:23.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:23.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:11:23.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:11:23.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:11:23.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:11:23.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:11:23.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:11:23.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:11:23.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:11:23.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:11:23.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:11:23.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:11:23.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:11:23.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:23.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:11:23.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.116337176, 2021-01-15T19:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.115966923, 2021-01-15T19:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.11563253, 2021-01-15T19:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 150.17437127100001, 2021-01-15T19:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.18200839900001, 2021-01-15T19:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.15201419499999, 2021-01-15T19:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:23.692 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:23.693 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:23.690+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:23.793 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:23.792+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:23.858 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:23.858 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:23.859 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:23.859 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:23.868 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:23.912 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:23.911+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:23.992 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:23.993 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:23.993 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:23.996 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:23.995+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:24.002 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:24.097 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:24.096+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:24.169 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:24.170 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:24.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:24.170 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:24.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:11:24.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:24.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.98252844000001, 2021-01-15T19:11:24.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:11:24.171 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:24.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:24.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:24.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:11:24.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:11:24.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:11:24.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:11:24.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:11:24.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:11:24.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:11:24.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:11:24.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:11:24.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:11:24.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:11:24.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:11:24.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:11:24.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:11:24.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:11:24.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:11:24.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:11:24.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.182 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.183 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:24.183 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:24.184 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.93504026299999, 2021-01-15T19:11:24.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:11:24.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:24.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.185 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.186 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.186 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:11:24.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.186 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:24.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:24.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:11:24.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:11:24.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:11:24.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:11:24.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:11:24.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:11:24.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:11:24.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:11:24.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:11:24.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:11:24.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:11:24.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:11:24.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:11:24.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:11:24.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:11:24.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:11:24.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:11:24.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:24.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:11:24.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:11:24.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:11:24.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:11:24.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.199 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.200 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:11:24.200 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:24.201 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 160.93489224499999, 2021-01-15T19:11:24.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:11:24.201 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.201 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.201 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.202 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:11:24.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.202 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:24.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:24.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:11:24.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:11:24.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:11:24.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:11:24.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:11:24.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:11:24.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:11:24.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:11:24.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:11:24.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:11:24.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:11:24.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:24.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:11:24.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:11:24.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:11:24.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:11:24.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:24.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:11:24.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:11:24.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:11:24.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:11:24.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:11:24.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:11:24.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:11:24.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:11:24.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.216 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.217 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:24.217 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:24.217 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.980571254, 2021-01-15T19:11:24.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:11:24.218 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.218 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.219 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.219 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:24.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.219 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:24.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:24.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:11:24.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:11:24.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:11:24.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:11:24.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:11:24.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:11:24.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:11:24.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:11:24.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:11:24.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:11:24.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:11:24.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:24.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:11:24.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:11:24.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:11:24.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:11:24.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:11:24.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:11:24.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:11:24.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:11:24.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.232 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:24.232 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:24.232 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.980379965, 2021-01-15T19:11:24.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:11:24.232 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.233 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.233 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.233 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.233 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.233 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:24.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.234 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:24.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:24.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:11:24.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:11:24.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:11:24.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:11:24.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:11:24.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:11:24.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:11:24.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:11:24.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:11:24.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:11:24.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:11:24.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:24.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:11:24.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:11:24.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:11:24.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:11:24.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:11:24.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:11:24.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:11:24.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:11:24.246 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.246 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.246 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.247 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:24.247 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:24.247 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 125.980217612, 2021-01-15T19:11:24.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:11:24.248 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.248 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.248 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.249 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.249 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:24.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.249 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:24.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:11:24.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:11:24.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:11:24.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:11:24.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:11:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:11:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:11:24.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:11:24.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:11:24.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:11:24.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:24.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:11:24.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:11:24.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:11:24.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:11:24.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:11:24.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:11:24.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:11:24.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:11:24.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.261 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.261 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:24.262 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:24.262 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.948497699, 2021-01-15T19:11:24.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:11:24.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.264 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:11:24.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:24.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:24.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:11:24.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:11:24.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:11:24.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:11:24.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:11:24.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:11:24.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:11:24.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:11:24.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:11:24.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:11:24.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:11:24.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:11:24.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:11:24.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:11:24.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:11:24.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.277 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:24.277 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:24.277 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 119.94843600999999, 2021-01-15T19:11:24.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:11:24.277 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.278 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.278 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.278 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:11:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:24.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:24.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:11:24.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:11:24.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:11:24.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:11:24.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:11:24.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:11:24.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:11:24.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:11:24.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:11:24.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:11:24.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:11:24.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:11:24.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:11:24.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:11:24.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:11:24.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.288 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:11:24.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:24.290 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.844369671999999, 2021-01-15T19:11:24.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:11:24.290 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.290 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:24.290 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.291 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:11:24.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.291 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:24.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:24.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:11:24.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:11:24.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:11:24.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:11:24.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:11:24.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:11:24.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:11:24.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:11:24.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:11:24.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:24.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:11:24.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:24.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:11:24.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:11:24.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:11:24.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:11:24.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:11:24.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:11:24.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:11:24.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:11:24.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:11:24.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:11:24.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:24.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:11:24.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:11:24.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:11:24.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:11:24.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.306 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.307 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:11:24.307 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:24.307 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 82.844283099999998, 2021-01-15T19:11:24.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:11:24.308 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.308 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.308 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.309 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:11:24.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:24.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:24.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:11:24.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:24.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:11:24.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:11:24.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:11:24.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:11:24.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:11:24.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:11:24.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:11:24.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:11:24.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:11:24.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:24.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:11:24.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:11:24.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:11:24.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:11:24.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:24.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:11:24.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:11:24.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:11:24.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:11:24.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:24.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:11:24.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:11:24.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:11:24.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:11:24.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:11:24.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:11:24.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:11:24.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:11:24.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.325 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:11:24.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:24.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.953972970999999, 2021-01-15T19:11:24.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:11:24.326 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.326 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:24.327 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:24.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:24.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:11:24.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:11:24.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:11:24.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:11:24.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:11:24.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:11:24.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:11:24.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:11:24.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:11:24.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:11:24.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:11:24.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:11:24.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:11:24.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:11:24.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:11:24.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:11:24.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:11:24.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.338 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:11:24.338 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:24.338 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.828812122000002, 2021-01-15T19:11:24.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:11:24.339 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.339 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:24.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.339 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.340 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.340 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:11:24.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.340 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:24.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:24.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:11:24.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:11:24.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:11:24.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:11:24.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:24.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:11:24.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:11:24.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:11:24.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:11:24.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:24.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:11:24.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:11:24.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:11:24.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:11:24.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:11:24.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:11:24.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:11:24.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:24.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:11:24.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:11:24.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:11:24.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:11:24.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.353 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.353 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.354 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:11:24.354 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:24.354 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 77.828716201999995, 2021-01-15T19:11:24.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:11:24.354 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.355 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.355 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.355 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:11:24.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.356 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:24.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:24.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:11:24.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:11:24.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:11:24.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:11:24.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:11:24.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:11:24.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:11:24.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:11:24.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:11:24.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:11:24.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:11:24.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:24.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:11:24.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:11:24.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:11:24.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:11:24.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:24.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:11:24.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:11:24.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:11:24.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:11:24.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:11:24.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:11:24.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:11:24.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:11:24.369 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.369 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.370 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.370 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:11:24.370 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:24.371 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.961657861999996, 2021-01-15T19:11:24.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:11:24.371 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.371 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:24.371 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.372 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.372 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:11:24.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.372 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:24.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:24.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:11:24.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:11:24.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:11:24.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:11:24.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:11:24.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:11:24.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:11:24.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:11:24.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:11:24.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:24.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:11:24.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:24.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:11:24.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:11:24.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:11:24.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:11:24.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:11:24.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:11:24.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:11:24.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:11:24.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:11:24.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:11:24.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:24.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:11:24.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:11:24.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:11:24.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:11:24.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.387 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.388 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:11:24.388 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:24.388 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 72.961566785000002, 2021-01-15T19:11:24.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:11:24.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.390 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:11:24.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.390 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:24.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:24.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:11:24.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:24.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:11:24.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:11:24.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:11:24.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:11:24.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:11:24.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:11:24.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:11:24.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:11:24.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:11:24.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:24.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:11:24.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:11:24.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:11:24.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:11:24.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:24.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:11:24.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:11:24.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:11:24.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:11:24.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:24.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:11:24.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:11:24.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:11:24.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:11:24.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:11:24.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:11:24.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:11:24.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:11:24.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.406 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.406 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:11:24.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:24.407 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.95208753, 2021-01-15T19:11:24.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:11:24.407 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.407 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:24.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.408 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:11:24.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.409 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:24.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:24.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:11:24.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:11:24.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:11:24.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:11:24.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:11:24.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:11:24.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:11:24.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:11:24.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:11:24.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:24.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:11:24.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:11:24.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:11:24.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:11:24.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:11:24.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:11:24.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:24.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:11:24.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:11:24.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:24.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:11:24.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:11:24.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:24.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:11:24.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:11:24.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:11:24.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:11:24.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.423 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.423 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:11:24.424 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:24.424 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.951942506000002, 2021-01-15T19:11:24.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:11:24.425 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.425 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.426 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:11:24.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.426 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:24.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:24.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:11:24.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:11:24.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:11:24.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:11:24.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:11:24.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:11:24.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:11:24.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:11:24.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:11:24.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:11:24.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:11:24.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:24.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:11:24.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:11:24.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:11:24.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:11:24.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:24.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:11:24.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:11:24.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:11:24.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:11:24.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:24.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:11:24.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:11:24.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:11:24.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:11:24.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:11:24.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:11:24.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:11:24.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:11:24.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.441 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.441 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:24.442 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:24.443 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.786448632999999, 2021-01-15T19:11:24.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:11:24.443 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.443 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:11:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:11:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:11:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:11:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:11:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:11:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:11:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:11:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:11:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:11:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:11:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:11:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:11:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:11:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:11:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:11:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.786361206000002, 2021-01-15T19:11:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:11:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:11:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:11:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:11:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:11:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:11:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:11:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:11:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:11:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:11:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:11:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:11:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:11:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:11:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:11:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:11:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:11:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.786281242000001, 2021-01-15T19:11:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:11:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:11:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:11:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:11:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:11:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:11:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:11:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:11:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:11:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:11:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:11:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:11:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:11:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:11:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:11:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:11:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:11:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:24.479 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:24.480 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:24.480 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:11:24.482 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:24.479+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:24.483 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:24.581 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:24.580+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:24.645 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:24.649 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:24.649 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:24.649 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:24.649 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:24.659 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:24.665 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:24.682 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:24.680+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:24.686 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:24.706 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:24.727 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:24.747 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:24.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:24.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:24.782 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:24.781+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:24.824 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:24.824 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:24.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:24.917 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:24.916+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:24.984 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:24.983+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:24.998 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:24.998 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.007 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:25.085 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:25.084+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:25.140 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:25.140 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:25.186 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:25.185+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:25.281 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:25.282 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.287 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:25.286+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:25.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:25.386 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:11:25.389 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:25.387+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:25.426 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:25.426 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:25.520 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:25.519+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:25.590 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:25.589+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:25.610 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:25.610 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:25.691 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:25.690+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:25.792 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:25.791+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:25.793 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:25.893 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:25.892+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:25.933 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:25.934 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:25.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:25.993 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:25.993+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:26.081 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:26.081 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:26.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:26.131 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:26.130+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:26.195 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:26.194+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:26.256 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:26.257 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:26.266 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:26.296 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:26.295+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:26.390 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:26.391 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:26.397 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:26.396+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:26.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:26.498 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:26.497+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:26.574 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:26.615 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:26.614+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:26.700 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:26.699+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:26.738 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:26.738 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:26.748 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:26.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:26.801 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:26.800+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:26.882 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:26.913 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:26.923 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:26.922+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:26.933 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:26.953 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:26.974 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:26.995 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:27.003 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:27.002+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:27.016 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:27.019 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:27.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:27.059 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:27.104 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:27.103+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:27.205 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:27.204+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:27.224 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:27.225 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:27.225 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:27.225 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:27.225 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:27.234 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:27.307 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:27.306+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:27.398 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:11:27.400 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:27.400 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:27.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:27.401 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:27.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.401 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:11:27.402 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:27.402 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.21369484799999, 2021-01-15T19:11:27.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:11:27.402 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.402 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:27.403 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:27.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:27.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:11:27.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:11:27.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:11:27.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:11:27.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:11:27.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:11:27.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:11:27.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:11:27.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:11:27.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:11:27.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:11:27.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:11:27.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:11:27.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:11:27.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:11:27.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:11:27.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:11:27.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.413 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.414 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:27.414 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:27.415 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.166206671, 2021-01-15T19:11:27.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:11:27.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:27.415 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.416 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.416 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:11:27.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.416 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:27.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:27.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:11:27.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:11:27.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:11:27.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:11:27.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:11:27.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:11:27.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:11:27.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:11:27.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:11:27.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:11:27.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:11:27.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:11:27.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:11:27.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:11:27.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:11:27.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:11:27.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:11:27.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:27.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:11:27.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:11:27.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:11:27.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:11:27.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.429 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.429 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.430 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:11:27.430 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:27.430 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.16605865299999, 2021-01-15T19:11:27.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:11:27.431 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.431 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.431 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.432 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:11:27.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.432 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:27.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:27.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:11:27.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:11:27.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:11:27.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:11:27.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:11:27.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:11:27.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:11:27.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:11:27.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:11:27.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:11:27.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:11:27.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:27.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:11:27.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:11:27.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:11:27.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:11:27.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:27.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:11:27.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:11:27.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:11:27.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:11:27.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:11:27.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:11:27.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:11:27.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.447 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:27.447 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:27.447 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 129.21173766199999, 2021-01-15T19:11:27.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:11:27.448 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.448 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.448 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.448 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.448 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.449 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:27.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.449 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:27.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:27.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:11:27.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:11:27.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:11:27.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:11:27.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:11:27.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:11:27.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:11:27.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:11:27.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:11:27.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:11:27.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:11:27.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:27.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:11:27.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:11:27.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:11:27.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:11:27.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:11:27.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:11:27.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:11:27.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:11:27.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.461 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:27.462 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:27.462 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 129.211546373, 2021-01-15T19:11:27.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:11:27.462 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.462 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.463 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.463 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:27.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:27.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:11:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:11:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:11:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:11:27.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:11:27.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:11:27.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:11:27.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:11:27.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:11:27.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:11:27.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:11:27.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:27.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:11:27.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:11:27.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:11:27.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:11:27.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:11:27.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:11:27.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:11:27.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:11:27.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.475 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.476 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:27.476 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:27.476 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 129.21138402, 2021-01-15T19:11:27.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:11:27.477 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.477 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.477 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.478 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:27.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.478 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:27.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:27.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:11:27.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:11:27.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:11:27.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:11:27.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:11:27.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:11:27.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:11:27.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:11:27.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:11:27.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:11:27.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:11:27.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:27.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:11:27.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:11:27.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:11:27.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:11:27.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:11:27.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:11:27.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:11:27.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:11:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 123.17966410699999, 2021-01-15T19:11:27.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:11:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:11:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:11:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:11:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:11:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:11:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:11:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:11:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:11:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:11:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:11:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:11:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:11:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:11:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:11:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:11:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:11:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.502 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:27.503 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 123.179602418, 2021-01-15T19:11:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:11:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.504 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:11:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:11:27.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:11:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:11:27.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:11:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:11:27.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:11:27.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:11:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:11:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:11:27.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:11:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:11:27.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:11:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:11:27.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:11:27.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:11:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:11:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.075536080000006, 2021-01-15T19:11:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:11:27.516 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.517 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:11:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:11:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:11:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:11:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:11:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:11:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:11:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:11:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:11:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:11:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:11:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:11:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:11:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:11:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:11:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:11:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:11:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:11:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:11:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:11:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:11:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:11:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:11:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:11:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:11:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.532 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:11:27.533 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 86.075449508000005, 2021-01-15T19:11:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:11:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.534 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:11:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:11:27.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:11:27.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:11:27.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:11:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:11:27.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:11:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:11:27.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:11:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:11:27.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:11:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:11:27.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:11:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:11:27.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:11:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:11:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:11:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:11:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:11:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:11:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:11:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:11:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:11:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:11:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:11:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:11:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:11:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:11:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.185139379000006, 2021-01-15T19:11:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:11:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:11:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:11:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:11:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:11:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:11:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:11:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:11:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:11:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:11:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:11:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:11:27.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:11:27.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 81.059978529999995, 2021-01-15T19:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:11:27.580 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 81.059882610000003, 2021-01-15T19:11:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:11:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:11:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.192824270000003, 2021-01-15T19:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 76.192733192999995, 2021-01-15T19:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.183253938, 2021-01-15T19:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.183108914000002, 2021-01-15T19:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.017615040999999, 2021-01-15T19:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:27.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:11:27.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:11:27.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:11:27.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:11:27.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:11:27.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:11:27.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:11:27.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:11:27.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:11:27.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:11:27.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:11:27.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:11:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:11:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:11:27.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:11:27.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.683 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:27.684 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:27.684 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.017527614000002, 2021-01-15T19:11:27.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:11:27.684 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.684 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.685 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:11:27.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.686 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:27.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:27.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:11:27.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:11:27.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:11:27.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:11:27.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:11:27.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:11:27.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:11:27.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:11:27.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:11:27.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:11:27.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:11:27.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:11:27.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:11:27.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:11:27.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:11:27.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.695 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.696 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:27.696 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:27.696 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.017447650000001, 2021-01-15T19:11:27.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:11:27.697 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:27.697 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:27.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:27.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:27.698 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:27.698 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:11:27.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.698 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:27.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:27.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:27.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:27.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:27.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:27.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:11:27.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:27.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:11:27.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:11:27.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:27.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:11:27.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:11:27.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:27.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:11:27.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:11:27.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:27.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:11:27.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:11:27.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:27.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:11:27.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:11:27.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:27.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:11:27.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:11:27.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:27.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:27.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:27.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:11:27.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:11:27.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:27.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:27.709 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:27.710 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:27.709+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:27.711 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:27.810 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:27.809+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:27.876 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:27.876 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:27.877 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:27.877 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:27.880 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7] out_osds: [5, 6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:27.881 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:27.881 INFO:tasks.thrashosds.thrasher:Adding osd 5 2021-01-15T19:11:27.881 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2021-01-15T19:11:27.886 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:27.927 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:27.926+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:28.017 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:28.016+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:28.052 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:28.054 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:28.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:28.123 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:28.122+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:28.214 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:28.213+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:28.217 INFO:tasks.ceph.mon.a.smithi001.stderr:2021-01-15T19:11:28.215+0000 7f9995812700 -1 mon.a@0(leader).osd e200 definitely_dead 0 2021-01-15T19:11:28.218 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:11:28.253 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:28.254 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:11:28.255 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:28.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:28.318 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:28.317+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:28.406 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:28.408 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:28.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:28.418 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:28.417+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:28.519 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:28.518+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:28.559 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:28.561 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:28.570 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:28.620 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:28.619+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:28.707 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 165.268293857, 2021-01-15T19:11:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:11:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:11:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:11:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:11:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:11:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:11:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:11:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:11:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:11:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:11:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:11:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:11:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:11:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:11:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:11:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:11:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:11:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:11:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:11:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:11:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:11:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:11:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:11:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:11:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:11:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:11:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:11:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:11:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:11:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:11:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 165.268154557, 2021-01-15T19:11:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:11:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:11:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:11:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:11:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:11:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:11:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:11:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:11:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:11:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:11:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:11:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:11:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:11:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:11:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:11:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:11:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:11:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:11:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:11:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:11:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:11:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:11:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:11:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:11:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:11:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:11:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:11:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:11:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:11:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:11:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:11:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 165.26803345100001, 2021-01-15T19:11:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:11:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:11:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:11:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:11:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:11:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:11:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:11:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:11:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:11:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:11:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:11:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:11:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:11:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:11:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:11:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:11:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:11:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:11:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:11:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:11:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:11:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:11:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:11:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:11:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:11:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:11:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:11:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:11:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:11:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:11:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:11:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 164.497752577, 2021-01-15T19:11:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:11:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:11:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:11:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:11:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:11:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:11:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:11:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:11:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:11:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:11:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:11:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:11:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:11:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:11:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:11:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:11:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:11:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:11:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:11:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:11:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:11:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:11:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:11:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:11:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:11:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:11:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:11:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 164.48877439899999, 2021-01-15T19:11:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:11:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:11:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:11:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:11:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:11:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:11:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:11:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:11:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:11:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:11:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:11:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:11:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:11:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:11:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:11:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:11:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:11:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:11:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:11:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:11:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:11:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:11:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:11:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:11:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:11:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 164.48825469900001, 2021-01-15T19:11:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:11:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:11:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:11:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:11:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:11:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:11:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:11:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:11:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:11:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:11:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:11:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:11:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:11:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:11:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:11:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:11:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:11:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:11:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:11:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:11:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:11:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:11:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:11:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:11:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:11:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:11:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:11:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 164.48657374499999, 2021-01-15T19:11:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:11:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:11:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:11:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:11:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:11:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:11:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:11:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:11:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:11:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:11:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:11:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:11:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:11:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:11:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:11:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:11:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:11:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:11:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:11:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:11:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:11:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:11:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:11:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:11:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:11:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:11:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 164.48644079100001, 2021-01-15T19:11:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:11:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:11:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:11:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:11:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:11:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:11:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:11:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:11:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:11:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:11:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:11:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:11:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:11:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:11:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:11:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:11:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:11:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:11:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:11:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:11:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:11:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:11:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:11:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:11:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:11:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:11:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 155.52390952100001, 2021-01-15T19:11:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:11:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:11:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:11:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:11:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:11:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:11:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:11:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:11:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:11:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:11:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:11:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:11:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:11:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:11:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:11:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:11:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:11:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:11:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:11:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:11:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:11:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:11:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:11:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:11:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:11:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:11:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:11:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:11:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 155.52374292799999, 2021-01-15T19:11:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:11:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:11:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:28.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:11:28.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:11:28.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:11:28.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:11:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:11:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:11:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:11:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:11:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:11:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:11:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:11:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:11:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:11:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:11:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:11:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:11:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:11:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:11:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:11:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:11:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:11:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:11:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:11:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:11:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:11:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:11:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:11:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.32520413399999, 2021-01-15T19:11:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:11:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:11:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:11:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:11:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:11:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:11:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:11:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:11:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:11:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:11:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:11:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:11:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:11:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:11:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:11:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:11:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:11:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:11:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:11:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:11:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:11:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:11:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:11:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:11:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:11:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:11:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:11:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.322934089, 2021-01-15T19:11:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:11:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:11:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:11:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:11:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:11:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:11:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:11:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:11:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:11:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:11:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:11:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:11:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:11:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:11:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:11:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:11:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:11:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:11:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:11:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:11:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:11:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:11:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:11:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:11:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:11:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:11:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:11:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.31839589800001, 2021-01-15T19:11:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:11:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:11:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:11:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:11:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:11:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:11:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:11:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:11:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:11:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:11:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:11:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:11:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:11:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:11:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:11:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:11:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:11:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:11:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:11:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:11:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:11:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:11:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:11:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:11:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:11:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:11:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.31581002600001, 2021-01-15T19:11:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:11:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:11:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:11:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:11:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:11:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:11:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:11:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:11:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:11:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:11:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:11:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:11:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:11:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:11:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:11:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:11:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:11:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:11:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:11:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:11:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:11:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:11:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:11:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:11:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:11:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:11:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:11:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.315435691, 2021-01-15T19:11:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:11:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:11:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:11:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:11:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:11:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:11:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:11:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:11:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:11:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:11:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:11:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:11:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:11:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:11:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:11:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:11:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:11:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:11:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:11:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:11:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:11:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:11:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:11:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:11:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:11:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:11:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:11:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 136.31529919600001, 2021-01-15T19:11:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:11:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:11:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:11:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:11:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:11:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:11:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:11:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:11:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:11:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:11:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:11:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:11:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:11:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:11:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:11:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:11:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:11:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:11:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:11:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:11:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:11:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:11:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:11:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:11:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:11:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:11:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 134.33290147599999, 2021-01-15T19:11:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:11:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:11:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:11:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:11:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:11:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:11:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:11:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:11:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:11:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:11:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:11:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:11:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:11:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:11:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:11:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:11:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:11:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:11:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:11:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:11:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:11:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:11:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:11:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:11:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:11:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:11:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.186148418000002, 2021-01-15T19:11:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:11:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:11:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:11:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:11:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:11:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:11:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:11:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:11:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:11:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:11:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:11:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:11:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:11:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:11:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:11:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:11:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:11:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:11:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:11:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:11:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:11:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:11:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:11:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:11:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 67.185985486000007, 2021-01-15T19:11:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:11:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:11:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:11:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:11:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:11:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:11:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:11:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:11:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:11:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:11:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:11:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:11:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:11:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:11:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:11:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:11:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:11:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:11:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:11:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:11:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:11:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:11:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:11:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:11:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:11:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:11:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 61.412312849000003, 2021-01-15T19:11:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:11:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:11:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:11:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:11:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:11:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:11:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:11:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:11:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:11:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:11:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:11:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:11:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:11:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:11:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:11:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:11:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:11:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:11:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:11:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:11:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:29.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:29.102 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:29.103 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:28.906+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:29.103 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:29.102+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:29.203 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:29.202+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:29.226 INFO:teuthology.orchestra.run.smithi001.stderr:marked in osd.5. 2021-01-15T19:11:29.239 INFO:tasks.thrashosds.thrasher:Added osd 5 2021-01-15T19:11:29.250 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:29.251 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:29.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:29.263 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:29.284 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:29.304 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:29.303+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:29.305 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:29.348 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:29.369 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:29.370 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:29.370 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:29.371 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:29.371 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:29.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:29.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:29.372 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:29.373 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:29.373 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:29.374 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:29.375 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:29.376 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:29.397 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:29.397 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:29.405 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:29.404+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:29.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:29.506 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:29.505+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:29.579 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:29.579 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:29.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:29.607 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:29.606+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:29.708 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:29.707+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:29.727 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:29.727 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:29.737 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:29.809 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:29.808+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:29.900 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:29.901 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:29.910 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:29.911 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:29.909+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:30.011 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:30.010+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:30.035 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:30.035 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:30.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:30.130 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:30.129+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:30.211 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:30.212 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:30.214 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:30.213+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:30.224 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:30.229 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:11:30.314 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:30.313+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:30.411 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:30.411 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:30.415 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:30.414+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:30.423 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:30.516 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:30.515+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:30.588 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:30.589 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:30.599 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:30.617 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:30.616+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:30.718 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:30.717+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:30.765 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:30.765 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:30.774 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:30.819 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:30.818+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:30.900 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:30.901 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:30.910 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:30.955 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:30.954+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:31.020 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:31.019+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:31.035 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:31.036 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:31.045 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:31.122 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:31.120+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:31.210 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:31.210 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:31.219 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:31.222 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:31.220+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:31.231 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:11:31.322 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:31.321+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:31.371 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:31.372 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:31.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:31.381 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:31.425 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:31.424+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:31.506 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:31.507 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:31.516 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:31.530 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:31.556 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:31.557 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:31.556+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:31.576 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:31.597 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:31.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:31.625 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:31.624+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:31.639 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:31.645 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:31.646 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:31.653 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:31.653 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:31.654 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:31.654 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:31.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:31.726 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:31.725+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:31.827 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:31.825+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:31.835 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:31.836 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:31.836 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:31.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:31.927 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:31.927+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:32.019 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:32.019 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:32.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:32.022 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:32.023 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.581334107, 2021-01-15T19:11:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:11:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:11:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:11:32.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:11:32.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:11:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:11:32.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:11:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:11:32.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:11:32.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:11:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:11:32.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:11:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:11:32.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:11:32.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:11:32.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:11:32.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:11:32.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:11:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:11:32.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:11:32.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:11:32.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:11:32.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:11:32.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:11:32.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:11:32.040 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:11:32.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.040 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.040 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:11:32.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:11:32.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.041 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.041 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.041 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:11:32.042 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:11:32.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.042 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.042 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.043 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:32.043 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:32.043 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.581194807, 2021-01-15T19:11:32.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:11:32.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.044 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.045 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:11:32.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.045 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:32.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:32.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:11:32.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:11:32.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:11:32.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:11:32.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:11:32.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:11:32.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:11:32.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:11:32.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:11:32.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:11:32.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:11:32.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:11:32.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:11:32.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:11:32.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:11:32.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:11:32.056 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:11:32.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.056 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.056 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:11:32.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:11:32.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.057 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.057 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:11:32.058 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:11:32.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.058 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.058 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:11:32.059 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:11:32.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:11:32.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:11:32.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:11:32.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:11:32.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:11:32.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:11:32.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.063 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:32.063 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:32.063 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.58107370100001, 2021-01-15T19:11:32.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:11:32.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.064 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.064 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.065 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:11:32.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.065 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:32.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:32.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:11:32.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:11:32.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:11:32.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:11:32.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:11:32.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:11:32.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:11:32.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:11:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:11:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:11:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:11:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:11:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:11:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.073 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:11:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:11:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:11:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:11:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.075 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:11:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:11:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:11:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:11:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:11:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:11:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:11:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:11:32.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:11:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:11:32.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:11:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:11:32.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.082 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 167.810792827, 2021-01-15T19:11:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:11:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:11:32.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:32.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:32.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:11:32.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:11:32.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:11:32.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:11:32.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:11:32.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:11:32.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:11:32.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:11:32.090 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:11:32.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.091 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.091 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:11:32.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:11:32.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.092 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.092 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:11:32.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:11:32.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.093 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.093 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.093 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:11:32.093 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:11:32.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.094 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:11:32.094 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:11:32.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:11:32.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:11:32.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:11:32.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:11:32.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:11:32.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:11:32.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.098 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:11:32.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:11:32.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.099 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.100 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.100 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:32.100 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 167.80181464899999, 2021-01-15T19:11:32.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:11:32.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.101 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:11:32.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.102 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:32.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:32.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:11:32.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:11:32.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:11:32.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.109 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.109 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:11:32.109 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:11:32.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.110 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.110 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:11:32.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:32.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.110 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.111 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.111 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:11:32.111 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:11:32.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:32.112 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:11:32.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:11:32.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:11:32.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:11:32.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:11:32.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:11:32.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:11:32.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:11:32.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:11:32.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:11:32.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:11:32.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:11:32.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:11:32.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:11:32.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:11:32.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.120 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.120 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:32.121 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 167.80129494900001, 2021-01-15T19:11:32.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:11:32.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.121 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:11:32.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.122 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.123 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.123 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:32.123 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.124 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.124 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.124 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:32.124 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:11:32.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.125 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.125 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.125 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:11:32.125 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:11:32.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.126 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.126 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.126 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:11:32.126 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:11:32.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.127 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.127 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.127 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:11:32.127 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:11:32.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.128 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.128 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:11:32.128 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:11:32.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.129 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.129 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:11:32.129 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:11:32.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.130 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.130 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.130 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:11:32.130 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:11:32.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.131 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.131 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.131 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:11:32.131 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:11:32.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.132 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.132 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:11:32.132 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:11:32.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.133 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.133 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:11:32.133 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:11:32.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.134 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.134 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:11:32.134 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:11:32.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.135 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.135 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:11:32.135 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:11:32.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.136 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.136 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:11:32.136 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:11:32.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 167.79961399499999, 2021-01-15T19:11:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:11:32.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:11:32.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.140 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.140 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.140 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:32.141 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.141 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.141 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:32.142 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:11:32.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.142 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.142 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:11:32.143 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:11:32.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.143 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.143 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:11:32.144 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:11:32.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.144 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.144 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:11:32.145 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:11:32.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.145 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.145 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:11:32.146 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:11:32.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.146 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.146 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:32.147 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:11:32.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.147 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.147 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:11:32.148 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:11:32.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.148 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.148 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:11:32.149 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:11:32.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.149 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.149 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:11:32.150 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:11:32.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.150 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.150 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:11:32.151 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:11:32.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.151 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.151 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:11:32.152 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:11:32.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.152 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.153 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:11:32.153 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:11:32.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.153 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.154 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:11:32.154 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:11:32.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.154 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.155 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:32.155 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:32.155 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 167.79948104100001, 2021-01-15T19:11:32.156 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:11:32.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.156 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.157 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:11:32.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.157 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.157 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.158 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.158 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:32.158 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:11:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:11:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:11:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.161 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.161 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:11:32.161 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:11:32.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.162 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.162 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:11:32.162 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:11:32.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.163 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.163 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:11:32.163 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:11:32.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.164 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.164 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:11:32.164 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:11:32.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.164 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.165 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:32.165 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:11:32.165 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:11:32.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.166 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.166 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:11:32.166 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:11:32.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.167 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.167 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:11:32.167 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:11:32.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.168 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.168 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:11:32.168 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:11:32.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.169 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.169 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:11:32.169 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:11:32.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.169 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:11:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:11:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.170 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.171 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.171 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.171 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:11:32.171 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:32.172 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 158.83694977100001, 2021-01-15T19:11:32.172 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:11:32.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.172 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.173 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.173 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:11:32.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.173 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.174 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.174 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.174 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:32.174 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.175 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.175 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.175 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:32.175 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:11:32.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.176 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.176 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:11:32.176 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:11:32.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.177 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.177 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.177 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:11:32.177 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:11:32.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.178 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.178 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.178 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:11:32.178 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:11:32.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.179 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.179 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:11:32.179 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:11:32.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.180 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.180 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:11:32.180 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:11:32.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.181 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:32.181 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:11:32.181 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:11:32.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.182 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.182 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.182 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:11:32.182 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:11:32.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.183 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.183 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.183 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:11:32.183 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:11:32.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.184 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.184 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.184 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:11:32.184 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:11:32.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.185 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.185 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.185 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:11:32.185 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:11:32.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.186 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.186 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:11:32.186 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:11:32.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.187 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.187 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:11:32.187 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:11:32.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.188 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.188 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:11:32.189 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:32.189 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 158.83678317799999, 2021-01-15T19:11:32.189 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:11:32.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.190 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.190 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.190 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:11:32.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.191 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.191 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.191 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.191 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:32.191 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.192 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.192 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.192 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:32.193 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:11:32.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.193 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.193 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:11:32.194 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:11:32.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.194 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.194 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.194 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:11:32.195 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:11:32.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.195 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.195 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:11:32.196 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:11:32.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.196 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.196 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:11:32.197 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:11:32.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.197 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.197 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:11:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:11:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:32.198 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:11:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:11:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.199 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:11:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:11:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:32.200 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:11:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:11:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.201 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:11:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:11:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.202 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:11:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:11:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.203 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:11:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:11:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.204 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:11:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:11:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.205 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:11:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:11:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.63824438399999, 2021-01-15T19:11:32.207 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:11:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:11:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.209 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.210 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.211 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.211 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:32.211 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:11:32.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.211 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.212 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.212 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:11:32.212 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:11:32.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.212 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.213 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.213 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:11:32.213 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:11:32.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.214 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.214 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:11:32.214 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:11:32.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.215 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.215 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:11:32.215 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:11:32.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.216 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.216 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:11:32.216 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:11:32.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.217 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.217 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:11:32.217 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:11:32.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.218 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.218 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:11:32.218 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:11:32.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:11:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:11:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.220 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.220 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:11:32.220 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:11:32.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.221 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.221 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:11:32.221 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:11:32.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.222 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.222 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:11:32.222 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:11:32.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.223 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.223 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:11:32.223 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:11:32.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.223 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.224 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:32.224 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.635974339, 2021-01-15T19:11:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:11:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.226 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:11:32.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.226 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.227 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.227 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:32.227 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.228 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.228 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:32.228 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:11:32.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.229 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.229 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:11:32.229 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:11:32.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.230 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.230 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:11:32.230 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:11:32.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.231 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.231 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.231 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:11:32.231 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:11:32.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.232 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.232 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.232 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:11:32.232 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:11:32.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.233 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.233 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.233 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:11:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:11:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:11:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:11:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:11:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:11:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:11:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:11:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:11:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:11:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:11:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:11:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:11:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:11:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:11:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:11:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.63143614800001, 2021-01-15T19:11:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:11:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:11:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:11:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:11:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:11:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.248 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.248 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.248 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:11:32.248 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:11:32.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.249 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.249 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:11:32.249 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:32.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.250 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.250 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.250 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:11:32.250 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:11:32.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.251 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.251 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.251 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:11:32.251 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:11:32.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.252 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.252 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.252 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:11:32.252 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:11:32.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.253 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.253 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.253 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:11:32.253 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:11:32.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.254 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.254 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.254 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:11:32.254 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:11:32.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.255 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.255 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:11:32.255 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:11:32.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.256 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.256 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.256 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:11:32.256 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:11:32.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.257 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.257 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:11:32.257 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:11:32.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.258 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.258 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:11:32.258 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:11:32.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.259 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.260 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:32.260 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:32.260 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.62885027600001, 2021-01-15T19:11:32.260 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:11:32.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.261 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:11:32.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.262 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.262 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.262 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.262 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:32.263 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.263 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.263 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.263 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:32.264 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:11:32.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.264 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.264 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.264 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:11:32.265 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:11:32.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.265 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.265 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.265 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:11:32.266 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:11:32.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.266 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.266 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.266 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:11:32.267 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:11:32.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.267 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.267 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.267 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:11:32.268 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:11:32.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.268 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.268 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:11:32.269 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:11:32.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.269 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.269 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:11:32.270 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:11:32.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.270 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.270 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.270 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:11:32.271 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:11:32.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.271 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.271 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.271 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:11:32.272 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:11:32.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.272 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.272 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:11:32.273 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:11:32.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.273 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.273 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.273 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:11:32.274 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:11:32.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.274 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.274 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:11:32.275 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:11:32.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.275 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.275 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.275 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:11:32.276 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:11:32.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.276 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.276 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.277 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:32.277 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:32.277 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.628475941, 2021-01-15T19:11:32.277 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:11:32.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.278 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:11:32.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.279 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.279 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.279 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.280 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:32.280 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.280 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.281 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:32.281 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:11:32.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.281 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.282 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.282 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:11:32.282 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:11:32.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.283 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.283 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:11:32.283 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:11:32.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.283 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.284 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.284 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:11:32.284 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:11:32.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.284 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.285 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:11:32.285 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:11:32.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.285 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.285 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.286 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:11:32.286 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:11:32.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.287 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.287 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:11:32.287 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:11:32.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.288 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.288 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:11:32.288 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:11:32.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.289 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.289 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:11:32.289 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:11:32.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.290 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.290 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:11:32.290 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:11:32.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.291 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.291 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:11:32.291 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:11:32.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.291 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.292 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.292 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:11:32.292 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:11:32.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.293 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.293 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:11:32.293 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:11:32.293 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.293 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.294 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:32.294 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:32.295 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 139.62833944600001, 2021-01-15T19:11:32.295 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:11:32.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.295 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.296 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:11:32.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.297 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.297 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.297 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:32.297 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.298 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.298 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:32.298 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:11:32.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.299 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.299 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:11:32.299 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:11:32.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.300 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.300 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:11:32.300 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:11:32.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.301 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.301 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:11:32.301 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:11:32.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.302 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.302 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:11:32.302 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:11:32.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.303 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.303 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:11:32.303 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:11:32.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.304 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.304 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:11:32.304 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:11:32.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.305 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.305 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.305 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:11:32.305 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:11:32.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.306 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.306 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.306 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:11:32.306 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:11:32.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.307 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.307 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:11:32.307 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:11:32.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.308 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.308 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:11:32.308 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:11:32.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.309 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.309 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:11:32.309 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:11:32.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.310 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.310 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.311 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:11:32.311 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:32.311 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 137.64594172599999, 2021-01-15T19:11:32.311 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:11:32.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.312 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:11:32.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.313 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.313 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.314 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:32.314 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.314 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.315 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:32.315 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:11:32.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.315 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.316 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:11:32.316 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:11:32.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.316 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.317 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:11:32.317 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:11:32.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.317 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.317 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.318 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:11:32.318 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:11:32.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.318 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.318 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.319 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:11:32.319 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:11:32.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.319 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.319 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.320 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:11:32.320 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:11:32.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.320 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.320 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:32.321 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:11:32.321 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:11:32.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.321 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.322 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:11:32.322 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:11:32.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.322 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.322 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:11:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:11:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.323 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:11:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:11:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.324 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:11:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:11:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.325 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.326 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:11:32.326 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:11:32.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.326 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.326 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.327 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.327 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:11:32.327 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:32.327 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.499188668000002, 2021-01-15T19:11:32.328 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:11:32.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.328 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.329 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:11:32.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.329 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.330 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.330 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:32.330 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.330 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.331 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.331 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:32.331 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:11:32.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.331 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.332 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.332 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:11:32.332 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:11:32.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.332 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.333 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.333 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:11:32.333 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:11:32.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.333 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.334 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.334 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:11:32.334 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:11:32.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.334 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.335 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.335 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:11:32.335 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:11:32.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:11:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:11:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.337 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.337 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:11:32.337 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:11:32.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.338 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.338 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:11:32.338 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:11:32.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.339 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.339 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.339 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:11:32.339 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:11:32.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.340 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.340 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:11:32.340 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:11:32.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.341 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.341 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:11:32.341 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:11:32.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.342 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.342 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:11:32.343 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:32.343 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 70.499025735999993, 2021-01-15T19:11:32.343 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:11:32.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.344 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:11:32.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.345 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.345 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.345 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.345 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:32.345 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.346 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.346 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.346 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:32.346 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:11:32.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.347 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.347 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:11:32.347 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:11:32.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.348 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.348 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.348 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:11:32.348 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:11:32.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.349 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.349 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.349 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:11:32.349 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:11:32.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.350 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.350 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.350 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:11:32.350 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:11:32.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.351 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.351 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:11:32.351 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:11:32.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.352 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.352 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:32.352 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:11:32.352 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:11:32.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.353 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.353 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:11:32.353 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:11:32.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.354 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.354 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.354 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:11:32.354 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:11:32.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.355 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.355 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:11:32.355 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:11:32.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.356 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.356 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.356 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:11:32.356 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:11:32.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.357 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.357 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:11:32.357 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:11:32.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.358 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.358 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:11:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 64.725353099000003, 2021-01-15T19:11:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:11:32.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.360 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.361 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:11:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.362 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:11:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:11:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.363 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:11:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:11:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.364 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:11:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:11:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.365 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:11:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:11:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.366 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:11:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:11:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:32.367 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:11:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:11:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.368 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:11:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:11:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.369 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:11:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:11:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.370 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:11:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:11:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.372 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.372 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:32.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:32.374 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:32.372+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:32.474 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:32.474+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:32.508 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:32.509 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:32.509 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:32.509 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:32.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:32.575 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:32.574+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:32.649 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:32.650 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:32.650 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:32.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:32.676 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:32.675+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:32.776 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:32.776+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:32.794 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:32.794 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:32.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 173.54319960699999, 2021-01-15T19:11:32.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:11:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.798 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:11:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.801 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.801 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:32.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:11:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:11:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:11:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:11:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:11:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:11:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:11:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:11:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:11:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:11:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:11:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:11:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:11:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:11:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:11:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:11:32.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:11:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:11:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:11:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:11:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:11:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 173.54300866299999, 2021-01-15T19:11:32.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:11:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.817 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:11:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.818 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.819 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:11:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:11:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:11:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:11:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:11:32.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:11:32.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:11:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:11:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:11:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:11:32.825 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:11:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:11:32.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:11:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:11:32.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:11:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:11:32.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:11:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:11:32.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:11:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:11:32.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:11:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:11:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:11:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:32.833 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.604056566, 2021-01-15T19:11:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:11:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.834 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:11:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:11:32.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:11:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:11:32.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.839 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:11:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:11:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.840 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:11:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:11:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.841 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:11:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:11:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.842 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:11:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:11:32.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:11:32.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:11:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:11:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:11:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:11:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:11:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:11:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:11:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:11:32.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:11:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.601336105, 2021-01-15T19:11:32.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:11:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.852 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:11:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.853 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:32.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:11:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:11:32.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:11:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.857 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:11:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:11:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:11:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:11:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:11:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:11:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:11:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:11:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:11:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:11:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:11:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:11:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:11:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:11:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:11:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:11:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:11:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:11:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:11:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:11:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.60122352600001, 2021-01-15T19:11:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:11:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:11:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:11:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:11:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:11:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:11:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:11:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:11:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:11:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:11:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:11:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:11:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:11:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:11:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:11:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:11:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:11:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:11:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:11:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:11:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:11:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:11:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:11:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.60109362200001, 2021-01-15T19:11:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:11:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:11:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:11:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:11:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:11:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:11:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:11:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:11:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:11:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:11:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:11:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:11:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:11:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:11:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:11:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:11:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:11:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:11:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:11:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:11:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:11:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:11:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:11:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:11:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:11:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.57788835299999, 2021-01-15T19:11:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:11:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:11:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:11:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:11:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:11:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:11:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:11:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:11:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:11:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:11:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:11:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:11:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:11:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:11:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:11:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:11:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:11:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:11:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:11:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:11:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:11:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:11:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.577770155, 2021-01-15T19:11:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:11:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:11:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:11:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:11:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:11:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:11:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:11:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:11:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:11:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:11:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:11:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:11:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:11:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:11:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:11:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:11:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:11:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:11:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:11:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:11:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:11:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:11:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:11:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.57765746499999, 2021-01-15T19:11:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:11:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:11:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:11:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:11:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:11:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:11:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:11:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:11:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:11:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:11:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:11:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:11:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:11:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:11:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:11:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:11:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:11:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:11:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:11:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:11:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:11:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:11:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:11:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.949 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:11:32.950 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.572055715, 2021-01-15T19:11:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:11:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:11:32.951 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:11:32.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220089+0000", 2021-01-15T19:11:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3100000000000006e-06 2021-01-15T19:11:32.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220098+0000", 2021-01-15T19:11:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5034999999999999e-05 2021-01-15T19:11:32.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220113+0000", 2021-01-15T19:11:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.917e-06 2021-01-15T19:11:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220120+0000", 2021-01-15T19:11:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023371151 2021-01-15T19:11:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243491+0000", 2021-01-15T19:11:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0896000000000001e-05 2021-01-15T19:11:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.243512+0000", 2021-01-15T19:11:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999 2021-01-15T19:11:32.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.960 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:567 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~548864] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.56358699099999, 2021-01-15T19:11:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131, 2021-01-15T19:11:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 567 2021-01-15T19:11:32.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:32.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228506+0000", 2021-01-15T19:11:32.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044842800000000001 2021-01-15T19:11:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228954+0000", 2021-01-15T19:11:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8289999999999997e-06 2021-01-15T19:11:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228963+0000", 2021-01-15T19:11:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4644000000000001e-05 2021-01-15T19:11:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.967 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228978+0000", 2021-01-15T19:11:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6100000000000002e-06 2021-01-15T19:11:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.968 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228984+0000", 2021-01-15T19:11:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00084452099999999999 2021-01-15T19:11:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.969 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229829+0000", 2021-01-15T19:11:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000127331 2021-01-15T19:11:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.229956+0000", 2021-01-15T19:11:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010883889999999999 2021-01-15T19:11:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.231044+0000", 2021-01-15T19:11:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.021693994000000001 2021-01-15T19:11:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252738+0000", 2021-01-15T19:11:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6731999999999995e-05 2021-01-15T19:11:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252835+0000", 2021-01-15T19:11:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000110653 2021-01-15T19:11:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252946+0000", 2021-01-15T19:11:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024440131 2021-01-15T19:11:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:568 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~622592] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.559068408, 2021-01-15T19:11:32.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739, 2021-01-15T19:11:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.977 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 568 2021-01-15T19:11:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233024+0000", 2021-01-15T19:11:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00036286000000000001 2021-01-15T19:11:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233387+0000", 2021-01-15T19:11:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0074999999999999e-05 2021-01-15T19:11:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233397+0000", 2021-01-15T19:11:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.56e-05 2021-01-15T19:11:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233413+0000", 2021-01-15T19:11:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3699999999999997e-06 2021-01-15T19:11:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233420+0000", 2021-01-15T19:11:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069021919999999997 2021-01-15T19:11:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.984 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240322+0000", 2021-01-15T19:11:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013187999999999999 2021-01-15T19:11:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.240454+0000", 2021-01-15T19:11:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001157768 2021-01-15T19:11:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.986 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.241612+0000", 2021-01-15T19:11:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011336977 2021-01-15T19:11:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252949+0000", 2021-01-15T19:11:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0059059469999999999 2021-01-15T19:11:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258855+0000", 2021-01-15T19:11:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2070000000000004e-05 2021-01-15T19:11:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.258947+0000", 2021-01-15T19:11:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.025922739 2021-01-15T19:11:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.990 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:571 3.9s0 3:940930a3:::smithi00140323-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:32.991 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 168.554050639, 2021-01-15T19:11:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373, 2021-01-15T19:11:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:32.992 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 571 2021-01-15T19:11:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.993 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:32.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238042+0000", 2021-01-15T19:11:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8541000000000003e-05 2021-01-15T19:11:32.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238081+0000", 2021-01-15T19:11:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0299999999999994e-06 2021-01-15T19:11:32.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238089+0000", 2021-01-15T19:11:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1705000000000001e-05 2021-01-15T19:11:32.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238100+0000", 2021-01-15T19:11:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.147e-06 2021-01-15T19:11:32.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.238105+0000", 2021-01-15T19:11:32.999 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.01409495 2021-01-15T19:11:33.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.000 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.000 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252200+0000", 2021-01-15T19:11:33.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108015 2021-01-15T19:11:33.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:33.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.252308+0000", 2021-01-15T19:11:33.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079878299999999996 2021-01-15T19:11:33.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260296+0000", 2021-01-15T19:11:33.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012681199999999999 2021-01-15T19:11:33.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260423+0000", 2021-01-15T19:11:33.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0013463430000000001 2021-01-15T19:11:33.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.261769+0000", 2021-01-15T19:11:33.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023727373 2021-01-15T19:11:33.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.005 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.006 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:696 3.9s0 3:940930a3:::smithi00140323-8:head [append 0~786432] snapc 2d=[2d,2a,28,26,25,24,22,1f,1b,19] ondisk+write+known_if_redirected e70)", 2021-01-15T19:11:33.006 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:33.006 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.58692656100001, 2021-01-15T19:11:33.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999, 2021-01-15T19:11:33.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.007 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 696 2021-01-15T19:11:33.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.008 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:33.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205166+0000", 2021-01-15T19:11:33.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00042303799999999998 2021-01-15T19:11:33.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205589+0000", 2021-01-15T19:11:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3191e-05 2021-01-15T19:11:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.012 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205602+0000", 2021-01-15T19:11:33.012 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9882999999999999e-05 2021-01-15T19:11:33.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.012 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.012 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.013 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205622+0000", 2021-01-15T19:11:33.013 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8109999999999992e-06 2021-01-15T19:11:33.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.013 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.014 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205631+0000", 2021-01-15T19:11:33.014 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.7403000000000004e-05 2021-01-15T19:11:33.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.014 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.014 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.015 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205708+0000", 2021-01-15T19:11:33.015 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.5789999999999993e-06 2021-01-15T19:11:33.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.015 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.016 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.205718+0000", 2021-01-15T19:11:33.016 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.002513904 2021-01-15T19:11:33.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.016 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208232+0000", 2021-01-15T19:11:33.017 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020869399999999999 2021-01-15T19:11:33.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.017 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.017 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.017 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.208441+0000", 2021-01-15T19:11:33.018 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017894689999999999 2021-01-15T19:11:33.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.018 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.018 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.019 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.210230+0000", 2021-01-15T19:11:33.019 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017951176999999999 2021-01-15T19:11:33.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.019 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.019 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.020 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228181+0000", 2021-01-15T19:11:33.020 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000146145 2021-01-15T19:11:33.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.020 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.020 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:33.021 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228327+0000", 2021-01-15T19:11:33.021 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.8147999999999997e-05 2021-01-15T19:11:33.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.021 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.022 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:58.228416+0000", 2021-01-15T19:11:33.022 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023249441999999999 2021-01-15T19:11:33.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.022 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.022 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.023 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.023 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:833 3.as0 3:5b6c0f9d:::smithi00140323-35:head [stat,delete] snapc 36=[36,34,2f,2a,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e80)", 2021-01-15T19:11:33.023 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:33.023 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 144.609739913, 2021-01-15T19:11:33.024 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999, 2021-01-15T19:11:33.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.024 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 833 2021-01-15T19:11:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.026 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.027 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.027 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182353+0000", 2021-01-15T19:11:33.027 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2129000000000003e-05 2021-01-15T19:11:33.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182395+0000", 2021-01-15T19:11:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1249e-05 2021-01-15T19:11:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.029 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.029 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182406+0000", 2021-01-15T19:11:33.029 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1699e-05 2021-01-15T19:11:33.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.030 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.030 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182428+0000", 2021-01-15T19:11:33.030 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3261000000000001e-05 2021-01-15T19:11:33.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.031 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.031 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182441+0000", 2021-01-15T19:11:33.031 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0717999999999995e-05 2021-01-15T19:11:33.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.031 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.032 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.032 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182522+0000", 2021-01-15T19:11:33.032 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3580000000000003e-06 2021-01-15T19:11:33.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.032 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.033 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.033 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.182530+0000", 2021-01-15T19:11:33.033 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003190072 2021-01-15T19:11:33.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.034 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.034 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185720+0000", 2021-01-15T19:11:33.034 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014350300000000001 2021-01-15T19:11:33.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.185864+0000", 2021-01-15T19:11:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001183396 2021-01-15T19:11:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.035 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.187047+0000", 2021-01-15T19:11:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015749459 2021-01-15T19:11:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.036 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.202797+0000", 2021-01-15T19:11:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000214367 2021-01-15T19:11:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.037 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.203011+0000", 2021-01-15T19:11:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0040111590000000003 2021-01-15T19:11:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.038 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:08.207022+0000", 2021-01-15T19:11:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024669369999999999 2021-01-15T19:11:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.039 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1202 3.as0 3:5dab47d9:::smithi00140323-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e130)", 2021-01-15T19:11:33.040 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 94.579192891999995, 2021-01-15T19:11:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343, 2021-01-15T19:11:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:33.041 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1202 2021-01-15T19:11:33.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:33.043 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212900+0000", 2021-01-15T19:11:33.044 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5281000000000003e-05 2021-01-15T19:11:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212935+0000", 2021-01-15T19:11:33.045 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0709999999999995e-06 2021-01-15T19:11:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212943+0000", 2021-01-15T19:11:33.046 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4119e-05 2021-01-15T19:11:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212957+0000", 2021-01-15T19:11:33.047 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.7729999999999999e-06 2021-01-15T19:11:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.212962+0000", 2021-01-15T19:11:33.048 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126279 2021-01-15T19:11:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213088+0000", 2021-01-15T19:11:33.049 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3668e-05 2021-01-15T19:11:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.213102+0000", 2021-01-15T19:11:33.050 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014565972 2021-01-15T19:11:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227668+0000", 2021-01-15T19:11:33.051 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012543799999999999 2021-01-15T19:11:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.227793+0000", 2021-01-15T19:11:33.052 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0068200930000000002 2021-01-15T19:11:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234613+0000", 2021-01-15T19:11:33.053 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014931999999999999 2021-01-15T19:11:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.234763+0000", 2021-01-15T19:11:33.054 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020263289999999999 2021-01-15T19:11:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:58.236789+0000", 2021-01-15T19:11:33.055 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023889343 2021-01-15T19:11:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1293 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 89.588534917000004, 2021-01-15T19:11:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757, 2021-01-15T19:11:33.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1293 2021-01-15T19:11:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.059 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.060 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203558+0000", 2021-01-15T19:11:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2899999999999999e-05 2021-01-15T19:11:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.061 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0003000000000001e-05 2021-01-15T19:11:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.062 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203611+0000", 2021-01-15T19:11:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9385000000000002e-05 2021-01-15T19:11:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.063 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203630+0000", 2021-01-15T19:11:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0978000000000001e-05 2021-01-15T19:11:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.064 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203641+0000", 2021-01-15T19:11:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032047500000000002 2021-01-15T19:11:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.065 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203961+0000", 2021-01-15T19:11:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4861000000000001e-05 2021-01-15T19:11:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.066 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203976+0000", 2021-01-15T19:11:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042877430000000001 2021-01-15T19:11:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.067 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208264+0000", 2021-01-15T19:11:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000179154 2021-01-15T19:11:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.068 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.208443+0000", 2021-01-15T19:11:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010789599999999999 2021-01-15T19:11:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.069 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.209522+0000", 2021-01-15T19:11:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023277552 2021-01-15T19:11:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.070 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.232800+0000", 2021-01-15T19:11:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026707 2021-01-15T19:11:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:33.071 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233067+0000", 2021-01-15T19:11:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000134676 2021-01-15T19:11:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.072 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233201+0000", 2021-01-15T19:11:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.029643757 2021-01-15T19:11:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.073 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1294 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 66] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:33.074 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 89.588395383999995, 2021-01-15T19:11:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002, 2021-01-15T19:11:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.075 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1294 2021-01-15T19:11:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.077 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203697+0000", 2021-01-15T19:11:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.3303999999999997e-05 2021-01-15T19:11:33.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203741+0000", 2021-01-15T19:11:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5159999999999997e-06 2021-01-15T19:11:33.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203748+0000", 2021-01-15T19:11:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3108000000000001e-05 2021-01-15T19:11:33.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203761+0000", 2021-01-15T19:11:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7470000000000002e-06 2021-01-15T19:11:33.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203767+0000", 2021-01-15T19:11:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030435 2021-01-15T19:11:33.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204071+0000", 2021-01-15T19:11:33.083 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2396e-05 2021-01-15T19:11:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204084+0000", 2021-01-15T19:11:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.019089892000000001 2021-01-15T19:11:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223174+0000", 2021-01-15T19:11:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000160519 2021-01-15T19:11:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223334+0000", 2021-01-15T19:11:33.086 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00049158999999999997 2021-01-15T19:11:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.223826+0000", 2021-01-15T19:11:33.087 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0093787850000000006 2021-01-15T19:11:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233204+0000", 2021-01-15T19:11:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024813399999999998 2021-01-15T19:11:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235686+0000", 2021-01-15T19:11:33.089 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114673 2021-01-15T19:11:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.090 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.235800+0000", 2021-01-15T19:11:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032103220000000002 2021-01-15T19:11:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1295 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 67=[67,66,64,63,60,5f,57,56,4b,44,26] ondisk+write+known_if_redirected e135)", 2021-01-15T19:11:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 89.588280071, 2021-01-15T19:11:33.092 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003, 2021-01-15T19:11:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.093 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1295 2021-01-15T19:11:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.094 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.095 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203813+0000", 2021-01-15T19:11:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2104000000000002e-05 2021-01-15T19:11:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.096 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203855+0000", 2021-01-15T19:11:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2496e-05 2021-01-15T19:11:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.097 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203867+0000", 2021-01-15T19:11:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2207e-05 2021-01-15T19:11:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203879+0000", 2021-01-15T19:11:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8319999999999996e-06 2021-01-15T19:11:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.099 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023940800000000001 2021-01-15T19:11:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204124+0000", 2021-01-15T19:11:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1895999999999999e-05 2021-01-15T19:11:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.101 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.204136+0000", 2021-01-15T19:11:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024577693000000001 2021-01-15T19:11:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.102 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228713+0000", 2021-01-15T19:11:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015151800000000001 2021-01-15T19:11:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.103 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.228865+0000", 2021-01-15T19:11:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0018583950000000001 2021-01-15T19:11:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.104 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.230723+0000", 2021-01-15T19:11:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0024829639999999998 2021-01-15T19:11:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.105 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.233206+0000", 2021-01-15T19:11:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0032156239999999998 2021-01-15T19:11:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236422+0000", 2021-01-15T19:11:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011398 2021-01-15T19:11:33.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:03.236536+0000", 2021-01-15T19:11:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032723117000000003 2021-01-15T19:11:33.108 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.109 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1735 3.as0 3:59c0717b:::smithi00140323-48:86 [read 0~4456448,checksum] snapc 0=[] ondisk+read+known_if_redirected e171)", 2021-01-15T19:11:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 53.474808775, 2021-01-15T19:11:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001, 2021-01-15T19:11:33.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1735 2021-01-15T19:11:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:33.112 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:33.113 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317284+0000", 2021-01-15T19:11:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4882000000000001e-05 2021-01-15T19:11:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:33.114 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317339+0000", 2021-01-15T19:11:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4389999999999997e-06 2021-01-15T19:11:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:33.115 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317345+0000", 2021-01-15T19:11:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1891e-05 2021-01-15T19:11:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:33.116 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317357+0000", 2021-01-15T19:11:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5580000000000002e-06 2021-01-15T19:11:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.117 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.317362+0000", 2021-01-15T19:11:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074883279999999998 2021-01-15T19:11:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:33.118 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324850+0000", 2021-01-15T19:11:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013976599999999999 2021-01-15T19:11:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:33.119 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.324990+0000", 2021-01-15T19:11:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016303502000000001 2021-01-15T19:11:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:33.120 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:39.341293+0000", 2021-01-15T19:11:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.024009366000000001 2021-01-15T19:11:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:33.122 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:33.122 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:33.123 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:33.124 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:33.122+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:33.224 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:33.223+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:33.240 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:33.239+0000 7ff42306b700 -1 osd.2 pg_epoch: 205 pg[3.4s0( v 153'82 (0'0,153'82] local-lis/les=204/205 n=2 ec=21/21 lis/c=204/204 les/c/f=205/205/0 sis=204) [2,5,4]p2(0) r=0 lpr=204 crt=153'82 lcod 153'81 mlcod 0'0 active+clean+snaptrim trimq=[4b~1,56~1,63~1,66~1,70~1,72~1,78~1,7d~1,84~1,8d~1,8f~1] ps=27] removing snap head 2021-01-15T19:11:33.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:11:33.271 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:33.272 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:33.272 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:33.273 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:33.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:33.325 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:33.324+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:33.426 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:33.425+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:33.448 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:33.448 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:33.449 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:33.449 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:33.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:33.542 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:33.541+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:33.625 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:33.625 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:33.626 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.43877286099999, 2021-01-15T19:11:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.627 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:33.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.629 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:33.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:33.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:33.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:33.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:33.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:33.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:33.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 145.43860992, 2021-01-15T19:11:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:33.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.643 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:33.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:33.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:33.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:33.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:33.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:33.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.660 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:33.661 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:33.661 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 141.27756727400001, 2021-01-15T19:11:33.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 141.27745981800001, 2021-01-15T19:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 141.144392952, 2021-01-15T19:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.24842526899999, 2021-01-15T19:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:33.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:33.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.758 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.758 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.759 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:33.759 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:33.759 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.24829513899999, 2021-01-15T19:11:33.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:33.760 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.760 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:33.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.761 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.761 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.761 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:33.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.762 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:33.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:33.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:33.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:33.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:33.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:33.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:33.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:33.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:33.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:33.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:33.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:33.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:33.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:33.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:33.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:33.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:33.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:33.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:33.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.773 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.774 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:33.774 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:33.774 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 136.24820973000001, 2021-01-15T19:11:33.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:33.775 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.775 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:33.775 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.776 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:33.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.776 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:33.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:33.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:33.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:33.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:33.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:33.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:33.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:33.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:33.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:33.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:33.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:33.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:33.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:33.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:33.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:33.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:33.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:33.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:33.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:33.789 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.41703534199999, 2021-01-15T19:11:33.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:33.790 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.790 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.791 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:33.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.791 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:33.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:33.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:33.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:33.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:33.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:33.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:33.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:33.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:33.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:33.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:33.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:33.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:33.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:33.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:33.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:33.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:33.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:33.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:33.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:33.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:33.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:33.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:33.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:33.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:33.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:33.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:33.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:33.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:33.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:33.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:33.811 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 130.416946693, 2021-01-15T19:11:33.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:33.811 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.812 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.812 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.812 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.812 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:33.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.813 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:33.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:33.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:33.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:33.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:33.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:33.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:33.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:33.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:33.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:33.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:33.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:33.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:33.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:33.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:33.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:33.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:33.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:33.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:33.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:33.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:33.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:33.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:33.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:33.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:33.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:33.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:33.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:33.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:33.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.433438635, 2021-01-15T19:11:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:33.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:33.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:33.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:33.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:33.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:33.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:33.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:33.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:33.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:33.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:33.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:33.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.851 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:33.852 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:33.852 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 120.43330656000001, 2021-01-15T19:11:33.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:33.852 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.852 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.853 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:33.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.854 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:33.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:33.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:33.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:33.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:33.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:33.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:33.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:33.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:33.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:33.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:33.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:33.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:33.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:33.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:33.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:33.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:33.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:33.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:33.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:33.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:33.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:33.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:33.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:33.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:33.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:33.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:33.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:33.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:33.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.871 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:33.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:33.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.430171043, 2021-01-15T19:11:33.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:33.873 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.873 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.874 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:33.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.874 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:33.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:33.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:33.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:33.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:33.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:33.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:33.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:33.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:33.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:33.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:33.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:33.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:33.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:33.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:33.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 115.430067017, 2021-01-15T19:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.419080268000002, 2021-01-15T19:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.418984746999996, 2021-01-15T19:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.418873601000001, 2021-01-15T19:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 66.327099278999995, 2021-01-15T19:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 66.325816140000001, 2021-01-15T19:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.420688171999998, 2021-01-15T19:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:34.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:34.013 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:34.011+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:34.013 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:34.113 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:34.111+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:34.166 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:34.180 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:34.181 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:34.181 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:34.186 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:34.190 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:34.207 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:34.228 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:34.234 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:34.233+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:34.239 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7, 5] out_osds: [6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:34.240 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:34.240 INFO:tasks.thrashosds.thrasher:Growing pool rbd 2021-01-15T19:11:34.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-01-15T19:11:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:11:34.260 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:34.281 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:34.282 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:34.282 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:34.282 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:34.282 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:34.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:34.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:34.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:34.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:34.283 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:34.284 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:34.284 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:34.285 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:34.321 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:34.321 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:34.326 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:34.325+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:34.331 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:34.415 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:34.414+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:34.498 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:34.498 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:34.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:34.548 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:34.547+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:34.559 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:11:34.560 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2021-01-15T19:11:34.572 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2021-01-15T19:11:34.239365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259129344,"num_objects":172,"num_object_clones":125,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":9,"num_read":610,"num_read_kb":726319,"num_write":1096,"num_write_kb":267684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":220061696,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3430,"ondisk_log_size":3430,"up":131,"acting":134,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":667500,"kb_used_data":397684,"kb_used_omap":0,"kb_used_meta":269248,"kb_avail":659935380,"statfs":{"total":676457349120,"available":675773829120,"internally_reserved":0,"allocated":407228416,"data_stored":718541437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":275709952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16998400,"num_objects":-13,"num_object_clones":-9,"num_object_copies":-39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":-1,"num_read":-33,"num_read_kb":-34934,"num_write":-56,"num_write_kb":-13577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":-7561216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022098"},"pg_stats":[{"pgid":"3.19","version":"176'70","reported_seq":"321","reported_epoch":"201","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:29.225780+0000","last_change":"2021-01-15T19:10:28.208323+0000","last_active":"2021-01-15T19:11:29.225780+0000","last_peered":"2021-01-15T19:11:29.225780+0000","last_clean":"2021-01-15T19:10:26.286623+0000","last_became_active":"2021-01-15T19:10:28.208323+0000","last_became_peered":"2021-01-15T19:10:28.208323+0000","last_unstale":"2021-01-15T19:11:29.225780+0000","last_undegraded":"2021-01-15T19:10:28.203425+0000","last_fullsized":"2021-01-15T19:10:28.203244+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'51","last_scrub_stamp":"2021-01-15T19:09:27.212108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:27.212108+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":38,"stat_sum":{"num_bytes":9764864,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":16204,"num_write":14,"num_write_kb":4360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2147483647],"acting":[4,7,2147483647],"avail_no_missing":["4(0)","7(1)"],"object_location_counts":[{"shards":"4(0),7(1)","objects":7}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":"1","reported_epoch":"202","state":"peering","last_fresh":"2021-01-15T19:11:30.225230+0000","last_change":"2021-01-15T19:11:30.225230+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:30.225230+0000","last_undegraded":"2021-01-15T19:11:30.225230+0000","last_fullsized":"2021-01-15T19:11:30.225230+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'72","last_scrub_stamp":"2021-01-15T19:10:14.380333+0000","last_deep_scrub":"51'34","last_deep_scrub_stamp":"2021-01-15T19:08:53.694318+0000","last_clean_scrub_stamp":"2021-01-15T19:10:14.380333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[2147483647,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":"2","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.230687+0000","last_change":"2021-01-15T19:11:29.230687+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:29.230687+0000","last_undegraded":"2021-01-15T19:11:29.230687+0000","last_fullsized":"2021-01-15T19:11:29.230687+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'30","last_scrub_stamp":"2021-01-15T19:09:25.976466+0000","last_deep_scrub":"50'30","last_deep_scrub_stamp":"2021-01-15T19:09:04.507336+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.976466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2147483647],"acting":[5,7,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"14'2","reported_seq":"152","reported_epoch":"203","state":"active+undersized+remapped","last_fresh":"2021-01-15T19:11:31.236986+0000","last_change":"2021-01-15T19:11:31.236859+0000","last_active":"2021-01-15T19:11:31.236986+0000","last_peered":"2021-01-15T19:11:31.236986+0000","last_clean":"2021-01-15T19:10:23.297118+0000","last_became_active":"2021-01-15T19:11:31.236859+0000","last_became_peered":"2021-01-15T19:11:31.236859+0000","last_unstale":"2021-01-15T19:11:31.236986+0000","last_undegraded":"2021-01-15T19:11:31.236986+0000","last_fullsized":"2021-01-15T19:11:31.229209+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[7],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"178'136","reported_seq":"559","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.223789+0000","last_change":"2021-01-15T19:10:46.778080+0000","last_active":"2021-01-15T19:11:29.223789+0000","last_peered":"2021-01-15T19:11:29.223789+0000","last_clean":"2021-01-15T19:11:29.223789+0000","last_became_active":"2021-01-15T19:09:15.343356+0000","last_became_peered":"2021-01-15T19:09:15.343356+0000","last_unstale":"2021-01-15T19:11:29.223789+0000","last_undegraded":"2021-01-15T19:11:29.223789+0000","last_fullsized":"2021-01-15T19:11:29.223789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":5,"last_scrub":"168'123","last_scrub_stamp":"2021-01-15T19:10:37.868333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:37.868333+0000","log_size":136,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7331840,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":15,"num_read_kb":16614,"num_write":19,"num_write_kb":7392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2621440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"3.15","version":"174'88","reported_seq":"466","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.220847+0000","last_change":"2021-01-15T19:10:45.262483+0000","last_active":"2021-01-15T19:10:45.262203+0000","last_peered":"2021-01-15T19:10:42.389278+0000","last_clean":"2021-01-15T19:10:26.827400+0000","last_became_active":"2021-01-15T19:10:28.207674+0000","last_became_peered":"2021-01-15T19:10:28.207674+0000","last_unstale":"2021-01-15T19:11:29.220847+0000","last_undegraded":"2021-01-15T19:11:29.220847+0000","last_fullsized":"2021-01-15T19:11:29.220847+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"71'32","last_scrub_stamp":"2021-01-15T19:09:03.911966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:03.911966+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9936896,"num_objects":6,"num_object_clones":5,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":2146,"num_write":51,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":5455872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2147483647,3],"acting":[2147483647,2147483647,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"1"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"3.14","version":"165'108","reported_seq":"403","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.221276+0000","last_change":"2021-01-15T19:10:45.264837+0000","last_active":"2021-01-15T19:10:45.264449+0000","last_peered":"2021-01-15T19:10:33.278808+0000","last_clean":"2021-01-15T19:10:26.284342+0000","last_became_active":"2021-01-15T19:10:28.212910+0000","last_became_peered":"2021-01-15T19:10:28.212910+0000","last_unstale":"2021-01-15T19:11:29.221276+0000","last_undegraded":"2021-01-15T19:11:29.221276+0000","last_fullsized":"2021-01-15T19:11:29.221276+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"148'94","last_scrub_stamp":"2021-01-15T19:10:16.897477+0000","last_deep_scrub":"148'94","last_deep_scrub_stamp":"2021-01-15T19:10:16.897477+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.897477+0000","log_size":108,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12705792,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":2353,"num_write":5,"num_write_kb":6008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,2147483647],"acting":[2147483647,2,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"1.11","version":"14'2","reported_seq":"150","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.224689+0000","last_change":"2021-01-15T19:10:28.206594+0000","last_active":"2021-01-15T19:11:29.224689+0000","last_peered":"2021-01-15T19:11:29.224689+0000","last_clean":"2021-01-15T19:10:23.286754+0000","last_became_active":"2021-01-15T19:10:28.206594+0000","last_became_peered":"2021-01-15T19:10:28.206594+0000","last_unstale":"2021-01-15T19:11:29.224689+0000","last_undegraded":"2021-01-15T19:11:29.224689+0000","last_fullsized":"2021-01-15T19:10:28.203753+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:55.119702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:55.119702+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":"413","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.220447+0000","last_change":"2021-01-15T19:10:46.270198+0000","last_active":"2021-01-15T19:11:29.220447+0000","last_peered":"2021-01-15T19:11:29.220447+0000","last_clean":"2021-01-15T19:10:44.257599+0000","last_became_active":"2021-01-15T19:10:46.270198+0000","last_became_peered":"2021-01-15T19:10:46.270198+0000","last_unstale":"2021-01-15T19:11:29.220447+0000","last_undegraded":"2021-01-15T19:11:29.220447+0000","last_fullsized":"2021-01-15T19:10:46.265074+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:37.482819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:09:18.175599+0000","last_clean_scrub_stamp":"2021-01-15T19:10:37.482819+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2147483647,2],"acting":[3,2147483647,2],"avail_no_missing":["3(0)","2(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"1.10","version":"14'2","reported_seq":"141","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.217702+0000","last_change":"2021-01-15T19:10:46.267032+0000","last_active":"2021-01-15T19:11:29.217702+0000","last_peered":"2021-01-15T19:11:29.217702+0000","last_clean":"2021-01-15T19:09:16.341317+0000","last_became_active":"2021-01-15T19:10:46.267032+0000","last_became_peered":"2021-01-15T19:10:46.267032+0000","last_unstale":"2021-01-15T19:11:29.217702+0000","last_undegraded":"2021-01-15T19:11:29.217702+0000","last_fullsized":"2021-01-15T19:10:46.265140+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:16.218565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.218565+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":"2","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.229663+0000","last_change":"2021-01-15T19:11:29.229663+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:29.229663+0000","last_undegraded":"2021-01-15T19:11:29.229663+0000","last_fullsized":"2021-01-15T19:11:29.229663+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'42","last_scrub_stamp":"2021-01-15T19:09:02.894120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.894120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2147483647,7],"acting":[5,2147483647,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"14'2","reported_seq":"169","reported_epoch":"202","state":"active+undersized","last_fresh":"2021-01-15T19:11:30.225331+0000","last_change":"2021-01-15T19:10:46.276115+0000","last_active":"2021-01-15T19:11:30.225331+0000","last_peered":"2021-01-15T19:11:30.225331+0000","last_clean":"2021-01-15T19:10:44.298014+0000","last_became_active":"2021-01-15T19:10:46.276115+0000","last_became_peered":"2021-01-15T19:10:46.276115+0000","last_unstale":"2021-01-15T19:11:30.225331+0000","last_undegraded":"2021-01-15T19:11:30.225331+0000","last_fullsized":"2021-01-15T19:10:46.269488+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"171'169","reported_seq":"605","reported_epoch":"201","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:29.226526+0000","last_change":"2021-01-15T19:10:28.208505+0000","last_active":"2021-01-15T19:11:29.226526+0000","last_peered":"2021-01-15T19:11:29.226526+0000","last_clean":"2021-01-15T19:10:26.810138+0000","last_became_active":"2021-01-15T19:10:28.208505+0000","last_became_peered":"2021-01-15T19:10:28.208505+0000","last_unstale":"2021-01-15T19:11:29.226526+0000","last_undegraded":"2021-01-15T19:10:28.205247+0000","last_fullsized":"2021-01-15T19:10:28.204970+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'62","last_scrub_stamp":"2021-01-15T19:09:14.332675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:14.332675+0000","log_size":169,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"stat_sum":{"num_bytes":14958592,"num_objects":11,"num_object_clones":9,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":17,"num_read_kb":18767,"num_write":59,"num_write_kb":12728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2147483647],"acting":[4,7,2147483647],"avail_no_missing":["4(0)","7(1)"],"object_location_counts":[{"shards":"4(0),7(1)","objects":11}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"1"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":"156","reported_epoch":"171","state":"stale+active+undersized","last_fresh":"2021-01-15T19:10:39.210985+0000","last_change":"2021-01-15T19:10:28.208168+0000","last_active":"2021-01-15T19:10:39.210985+0000","last_peered":"2021-01-15T19:10:39.210985+0000","last_clean":"2021-01-15T19:10:26.285035+0000","last_became_active":"2021-01-15T19:10:28.208168+0000","last_became_peered":"2021-01-15T19:10:28.208168+0000","last_unstale":"2021-01-15T19:10:45.259569+0000","last_undegraded":"2021-01-15T19:10:39.210985+0000","last_fullsized":"2021-01-15T19:10:28.204609+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:18.960545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.960545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"64'3","reported_seq":"204","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.220048+0000","last_change":"2021-01-15T19:10:38.897698+0000","last_active":"2021-01-15T19:11:29.220048+0000","last_peered":"2021-01-15T19:11:29.220048+0000","last_clean":"2021-01-15T19:11:29.220048+0000","last_became_active":"2021-01-15T19:10:23.277584+0000","last_became_peered":"2021-01-15T19:10:23.277584+0000","last_unstale":"2021-01-15T19:11:29.220048+0000","last_undegraded":"2021-01-15T19:11:29.220048+0000","last_fullsized":"2021-01-15T19:11:29.220048+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:10:38.897630+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:10:38.897630+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"214","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.225352+0000","last_change":"2021-01-15T19:10:46.279954+0000","last_active":"2021-01-15T19:11:29.225352+0000","last_peered":"2021-01-15T19:11:29.225352+0000","last_clean":"2021-01-15T19:10:44.261227+0000","last_became_active":"2021-01-15T19:10:46.279954+0000","last_became_peered":"2021-01-15T19:10:46.279954+0000","last_unstale":"2021-01-15T19:11:29.225352+0000","last_undegraded":"2021-01-15T19:11:29.225352+0000","last_fullsized":"2021-01-15T19:10:46.273505+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.103280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.103280+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"167'88","reported_seq":"583","reported_epoch":"201","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:29.224769+0000","last_change":"2021-01-15T19:10:28.210615+0000","last_active":"2021-01-15T19:11:29.224769+0000","last_peered":"2021-01-15T19:11:29.224769+0000","last_clean":"2021-01-15T19:10:26.814405+0000","last_became_active":"2021-01-15T19:10:28.210615+0000","last_became_peered":"2021-01-15T19:10:28.210615+0000","last_unstale":"2021-01-15T19:11:29.224769+0000","last_undegraded":"2021-01-15T19:10:28.204203+0000","last_fullsized":"2021-01-15T19:10:28.203981+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'37","last_scrub_stamp":"2021-01-15T19:08:54.947352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:54.947352+0000","log_size":88,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"stat_sum":{"num_bytes":3792896,"num_objects":4,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":36510,"num_write":17,"num_write_kb":3760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":8904704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2147483647],"acting":[4,7,2147483647],"avail_no_missing":["4(0)","7(1)"],"object_location_counts":[{"shards":"4(0),7(1)","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"1.6","version":"14'2","reported_seq":"154","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:30.224436+0000","last_change":"2021-01-15T19:10:51.158053+0000","last_active":"2021-01-15T19:11:30.224436+0000","last_peered":"2021-01-15T19:11:30.224436+0000","last_clean":"2021-01-15T19:11:30.224436+0000","last_became_active":"2021-01-15T19:10:23.284683+0000","last_became_peered":"2021-01-15T19:10:23.284683+0000","last_unstale":"2021-01-15T19:11:30.224436+0000","last_undegraded":"2021-01-15T19:11:30.224436+0000","last_fullsized":"2021-01-15T19:11:30.224436+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:51.157977+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:19.037889+0000","last_clean_scrub_stamp":"2021-01-15T19:10:51.157977+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"153'82","reported_seq":"577","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.222313+0000","last_change":"2021-01-15T19:11:30.223289+0000","last_active":"2021-01-15T19:11:29.220678+0000","last_peered":"2021-01-15T19:11:28.214506+0000","last_clean":"2021-01-15T19:10:26.287875+0000","last_became_active":"2021-01-15T19:10:28.209565+0000","last_became_peered":"2021-01-15T19:10:28.209565+0000","last_unstale":"2021-01-15T19:11:30.222313+0000","last_undegraded":"2021-01-15T19:11:30.222313+0000","last_fullsized":"2021-01-15T19:11:30.222313+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'69","last_scrub_stamp":"2021-01-15T19:10:01.942460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.942460+0000","log_size":82,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":622592,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":7,"num_read_kb":1825,"num_write":4,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,2147483647,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"},{"start":"79","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":"204","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.220915+0000","last_change":"2021-01-15T19:10:16.619367+0000","last_active":"2021-01-15T19:11:29.220915+0000","last_peered":"2021-01-15T19:11:29.220915+0000","last_clean":"2021-01-15T19:11:29.220915+0000","last_became_active":"2021-01-15T19:08:35.215541+0000","last_became_peered":"2021-01-15T19:08:35.215541+0000","last_unstale":"2021-01-15T19:11:29.220915+0000","last_undegraded":"2021-01-15T19:11:29.220915+0000","last_fullsized":"2021-01-15T19:11:29.220915+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:16.619291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:55.065810+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.619291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'2","reported_seq":"232","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.225028+0000","last_change":"2021-01-15T19:10:28.206487+0000","last_active":"2021-01-15T19:11:29.225028+0000","last_peered":"2021-01-15T19:11:29.225028+0000","last_clean":"2021-01-15T19:10:26.287284+0000","last_became_active":"2021-01-15T19:10:28.206487+0000","last_became_peered":"2021-01-15T19:10:28.206487+0000","last_unstale":"2021-01-15T19:11:29.225028+0000","last_undegraded":"2021-01-15T19:11:29.225028+0000","last_fullsized":"2021-01-15T19:10:28.204290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:28.194883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.194883+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1","version":"178'127","reported_seq":"596","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.221647+0000","last_change":"2021-01-15T19:11:30.222255+0000","last_active":"2021-01-15T19:11:30.221647+0000","last_peered":"2021-01-15T19:10:46.281470+0000","last_clean":"2021-01-15T19:10:46.281470+0000","last_became_active":"2021-01-15T19:09:51.207663+0000","last_became_peered":"2021-01-15T19:09:51.207663+0000","last_unstale":"2021-01-15T19:11:30.221647+0000","last_undegraded":"2021-01-15T19:11:30.221647+0000","last_fullsized":"2021-01-15T19:11:30.221647+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'114","last_scrub_stamp":"2021-01-15T19:10:28.056100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:28.056100+0000","log_size":127,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1581056,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":42,"num_write_kb":7408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":7340032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[2147483647,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5,7],"up_primary":5,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":"237","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:30.233286+0000","last_change":"2021-01-15T19:11:30.233286+0000","last_active":"2021-01-15T19:11:30.233286+0000","last_peered":"2021-01-15T19:11:30.233286+0000","last_clean":"2021-01-15T19:11:30.233286+0000","last_became_active":"2021-01-15T19:11:30.232922+0000","last_became_peered":"2021-01-15T19:11:30.232922+0000","last_unstale":"2021-01-15T19:11:30.233286+0000","last_undegraded":"2021-01-15T19:11:30.233286+0000","last_fullsized":"2021-01-15T19:11:30.233286+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:11:18.254551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:11:18.254551+0000","last_clean_scrub_stamp":"2021-01-15T19:11:18.254551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'18","reported_seq":"503","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.222338+0000","last_change":"2021-01-15T19:10:46.275951+0000","last_active":"2021-01-15T19:11:29.222338+0000","last_peered":"2021-01-15T19:11:29.222338+0000","last_clean":"2021-01-15T19:10:44.260749+0000","last_became_active":"2021-01-15T19:10:46.275951+0000","last_became_peered":"2021-01-15T19:10:46.275951+0000","last_unstale":"2021-01-15T19:11:29.222338+0000","last_undegraded":"2021-01-15T19:11:29.222338+0000","last_fullsized":"2021-01-15T19:10:46.271044+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'18","last_scrub_stamp":"2021-01-15T19:09:16.308952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.308952+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,2147483647],"acting":[4,2,2147483647],"avail_no_missing":["4(0)","2(1)"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"1.0","version":"14'2","reported_seq":"151","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.221683+0000","last_change":"2021-01-15T19:11:30.222910+0000","last_active":"2021-01-15T19:11:30.221683+0000","last_peered":"2021-01-15T19:11:28.213072+0000","last_clean":"2021-01-15T19:09:16.341317+0000","last_became_active":"2021-01-15T19:10:28.206057+0000","last_became_peered":"2021-01-15T19:10:28.206057+0000","last_unstale":"2021-01-15T19:11:30.221683+0000","last_undegraded":"2021-01-15T19:11:30.221683+0000","last_fullsized":"2021-01-15T19:11:30.221683+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:16.218565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:16.218565+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":5,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"204","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.220622+0000","last_change":"2021-01-15T19:10:28.526252+0000","last_active":"2021-01-15T19:11:29.220622+0000","last_peered":"2021-01-15T19:11:29.220622+0000","last_clean":"2021-01-15T19:11:29.220622+0000","last_became_active":"2021-01-15T19:08:35.211465+0000","last_became_peered":"2021-01-15T19:08:35.211465+0000","last_unstale":"2021-01-15T19:11:29.220622+0000","last_undegraded":"2021-01-15T19:11:29.220622+0000","last_fullsized":"2021-01-15T19:11:29.220622+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:28.526124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:28.526124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":"35","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.222503+0000","last_change":"2021-01-15T19:10:46.279939+0000","last_active":"2021-01-15T19:11:29.222503+0000","last_peered":"2021-01-15T19:11:29.222503+0000","last_clean":"2021-01-15T19:08:47.197945+0000","last_became_active":"2021-01-15T19:10:46.279939+0000","last_became_peered":"2021-01-15T19:10:46.279939+0000","last_unstale":"2021-01-15T19:11:29.222503+0000","last_undegraded":"2021-01-15T19:11:29.222503+0000","last_fullsized":"2021-01-15T19:10:46.273358+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:42.893436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:42.893436+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"178'116","reported_seq":"592","reported_epoch":"203","state":"active+undersized+remapped","last_fresh":"2021-01-15T19:11:31.330974+0000","last_change":"2021-01-15T19:11:31.331506+0000","last_active":"2021-01-15T19:11:31.330974+0000","last_peered":"2021-01-15T19:11:31.330974+0000","last_clean":"2021-01-15T19:10:46.311457+0000","last_became_active":"2021-01-15T19:11:31.237423+0000","last_became_peered":"2021-01-15T19:11:31.237423+0000","last_unstale":"2021-01-15T19:11:31.330974+0000","last_undegraded":"2021-01-15T19:11:31.330974+0000","last_fullsized":"2021-01-15T19:11:31.228666+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"174'106","last_scrub_stamp":"2021-01-15T19:10:43.182007+0000","last_deep_scrub":"174'106","last_deep_scrub_stamp":"2021-01-15T19:10:43.182007+0000","last_clean_scrub_stamp":"2021-01-15T19:10:43.182007+0000","log_size":116,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4374528,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":4707,"num_write":13,"num_write_kb":4272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":7454720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[2147483647,7,2],"avail_no_missing":["7(1)","2(2)","4(0)","5(0)"],"object_location_counts":[{"shards":"2(2),7(1)","objects":3}],"blocked_by":[],"up_primary":5,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":"241","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:30.239276+0000","last_change":"2021-01-15T19:11:30.236196+0000","last_active":"2021-01-15T19:11:30.239276+0000","last_peered":"2021-01-15T19:11:30.239276+0000","last_clean":"2021-01-15T19:11:30.239276+0000","last_became_active":"2021-01-15T19:11:30.235892+0000","last_became_peered":"2021-01-15T19:11:30.235892+0000","last_unstale":"2021-01-15T19:11:30.239276+0000","last_undegraded":"2021-01-15T19:11:30.239276+0000","last_fullsized":"2021-01-15T19:11:30.239276+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:11:14.077164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:10:05.030822+0000","last_clean_scrub_stamp":"2021-01-15T19:11:14.077164+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"14'2","reported_seq":"156","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.221633+0000","last_change":"2021-01-15T19:11:30.221871+0000","last_active":"2021-01-15T19:11:29.226677+0000","last_peered":"2021-01-15T19:11:28.218600+0000","last_clean":"2021-01-15T19:10:26.287136+0000","last_became_active":"2021-01-15T19:10:28.206697+0000","last_became_peered":"2021-01-15T19:10:28.206697+0000","last_unstale":"2021-01-15T19:11:30.221633+0000","last_undegraded":"2021-01-15T19:11:30.221633+0000","last_fullsized":"2021-01-15T19:11:30.221633+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:55.119702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:55.119702+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":"118","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.220127+0000","last_change":"2021-01-15T19:10:46.276794+0000","last_active":"2021-01-15T19:11:29.220127+0000","last_peered":"2021-01-15T19:11:29.220127+0000","last_clean":"2021-01-15T19:08:39.261385+0000","last_became_active":"2021-01-15T19:10:46.276794+0000","last_became_peered":"2021-01-15T19:10:46.276794+0000","last_unstale":"2021-01-15T19:11:29.220127+0000","last_undegraded":"2021-01-15T19:11:29.220127+0000","last_fullsized":"2021-01-15T19:10:46.265907+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:39.906692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:39.906692+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2,4],"acting":[2147483647,2,4],"avail_no_missing":["2(1)","4(2)"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":"121","reported_epoch":"202","state":"active+undersized","last_fresh":"2021-01-15T19:11:30.225804+0000","last_change":"2021-01-15T19:10:28.201710+0000","last_active":"2021-01-15T19:11:30.225804+0000","last_peered":"2021-01-15T19:11:30.225804+0000","last_clean":"2021-01-15T19:08:33.956428+0000","last_became_active":"2021-01-15T19:10:28.201710+0000","last_became_peered":"2021-01-15T19:10:28.201710+0000","last_unstale":"2021-01-15T19:11:30.225804+0000","last_undegraded":"2021-01-15T19:11:30.225804+0000","last_fullsized":"2021-01-15T19:10:28.200024+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:54.187515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:54.187515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"14'2","reported_seq":"170","reported_epoch":"171","state":"stale+active+undersized","last_fresh":"2021-01-15T19:10:39.210822+0000","last_change":"2021-01-15T19:10:28.207566+0000","last_active":"2021-01-15T19:10:39.210822+0000","last_peered":"2021-01-15T19:10:39.210822+0000","last_clean":"2021-01-15T19:09:22.199740+0000","last_became_active":"2021-01-15T19:10:28.207566+0000","last_became_peered":"2021-01-15T19:10:28.207566+0000","last_unstale":"2021-01-15T19:10:45.259577+0000","last_undegraded":"2021-01-15T19:10:39.210822+0000","last_fullsized":"2021-01-15T19:10:28.204264+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":3,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"178'164","reported_seq":"697","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.219907+0000","last_change":"2021-01-15T19:11:26.204837+0000","last_active":"2021-01-15T19:11:29.219907+0000","last_peered":"2021-01-15T19:11:29.219907+0000","last_clean":"2021-01-15T19:11:29.219907+0000","last_became_active":"2021-01-15T19:09:51.202675+0000","last_became_peered":"2021-01-15T19:09:51.202675+0000","last_unstale":"2021-01-15T19:11:29.219907+0000","last_undegraded":"2021-01-15T19:11:29.219907+0000","last_fullsized":"2021-01-15T19:11:29.219907+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"178'164","last_scrub_stamp":"2021-01-15T19:11:26.204742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:11:26.204742+0000","log_size":164,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14041088,"num_objects":8,"num_object_clones":6,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":22,"num_read_kb":26000,"num_write":22,"num_write_kb":13712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12623872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:30.231294+0000","last_change":"2021-01-15T19:11:30.231294+0000","last_active":"2021-01-15T19:11:30.231294+0000","last_peered":"2021-01-15T19:11:30.231294+0000","last_clean":"2021-01-15T19:11:30.231294+0000","last_became_active":"2021-01-15T19:11:30.230895+0000","last_became_peered":"2021-01-15T19:11:30.230895+0000","last_unstale":"2021-01-15T19:11:30.231294+0000","last_undegraded":"2021-01-15T19:11:30.231294+0000","last_fullsized":"2021-01-15T19:11:30.231294+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:11:16.172034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:11:16.172034+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"14'2","reported_seq":"200","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.221428+0000","last_change":"2021-01-15T19:10:28.205674+0000","last_active":"2021-01-15T19:11:29.221428+0000","last_peered":"2021-01-15T19:11:29.221428+0000","last_clean":"2021-01-15T19:09:22.199641+0000","last_became_active":"2021-01-15T19:10:28.205674+0000","last_became_peered":"2021-01-15T19:10:28.205674+0000","last_unstale":"2021-01-15T19:11:29.221428+0000","last_undegraded":"2021-01-15T19:11:29.221428+0000","last_fullsized":"2021-01-15T19:10:28.199783+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:07.176419+0000","last_clean_scrub_stamp":"2021-01-15T19:10:07.176419+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"178'382","reported_seq":"844","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.226573+0000","last_change":"2021-01-15T19:10:47.292589+0000","last_active":"2021-01-15T19:11:29.226573+0000","last_peered":"2021-01-15T19:11:29.226573+0000","last_clean":"2021-01-15T19:11:29.226573+0000","last_became_active":"2021-01-15T19:08:41.227078+0000","last_became_peered":"2021-01-15T19:08:41.227078+0000","last_unstale":"2021-01-15T19:11:29.226573+0000","last_undegraded":"2021-01-15T19:11:29.226573+0000","last_fullsized":"2021-01-15T19:11:29.226573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'350","last_scrub_stamp":"2021-01-15T19:10:42.241708+0000","last_deep_scrub":"173'350","last_deep_scrub_stamp":"2021-01-15T19:10:42.241708+0000","last_clean_scrub_stamp":"2021-01-15T19:10:42.241708+0000","log_size":382,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25812992,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":33,"num_read_kb":92991,"num_write":87,"num_write_kb":20560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":"175","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.227764+0000","last_change":"2021-01-15T19:10:46.280969+0000","last_active":"2021-01-15T19:11:29.227764+0000","last_peered":"2021-01-15T19:11:29.227764+0000","last_clean":"2021-01-15T19:10:44.260616+0000","last_became_active":"2021-01-15T19:10:46.280969+0000","last_became_peered":"2021-01-15T19:10:46.280969+0000","last_unstale":"2021-01-15T19:11:29.227764+0000","last_undegraded":"2021-01-15T19:11:29.227764+0000","last_fullsized":"2021-01-15T19:10:46.272483+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:25.339499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:25.339499+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.b","version":"14'2","reported_seq":"245","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.223464+0000","last_change":"2021-01-15T19:10:31.174783+0000","last_active":"2021-01-15T19:11:29.223464+0000","last_peered":"2021-01-15T19:11:29.223464+0000","last_clean":"2021-01-15T19:11:29.223464+0000","last_became_active":"2021-01-15T19:10:28.212262+0000","last_became_peered":"2021-01-15T19:10:28.212262+0000","last_unstale":"2021-01-15T19:11:29.223464+0000","last_undegraded":"2021-01-15T19:11:29.223464+0000","last_fullsized":"2021-01-15T19:11:29.223464+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:31.174667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:31.174667+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"178'273","reported_seq":"785","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:30.227881+0000","last_change":"2021-01-15T19:10:46.783556+0000","last_active":"2021-01-15T19:11:30.227881+0000","last_peered":"2021-01-15T19:11:30.227881+0000","last_clean":"2021-01-15T19:11:30.227881+0000","last_became_active":"2021-01-15T19:09:15.341093+0000","last_became_peered":"2021-01-15T19:09:15.341093+0000","last_unstale":"2021-01-15T19:11:30.227881+0000","last_undegraded":"2021-01-15T19:11:30.227881+0000","last_fullsized":"2021-01-15T19:11:30.227881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'253","last_scrub_stamp":"2021-01-15T19:10:39.050737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:39.050737+0000","log_size":273,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24305664,"num_objects":12,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":53,"num_read_kb":97876,"num_write":97,"num_write_kb":21032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"4"},{"start":"78","length":"2"},{"start":"7d","length":"1"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.8","version":"0'0","reported_seq":"33","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.217894+0000","last_change":"2021-01-15T19:10:46.269448+0000","last_active":"2021-01-15T19:11:29.217894+0000","last_peered":"2021-01-15T19:11:29.217894+0000","last_clean":"2021-01-15T19:08:35.214788+0000","last_became_active":"2021-01-15T19:10:46.269448+0000","last_became_peered":"2021-01-15T19:10:46.269448+0000","last_unstale":"2021-01-15T19:11:29.217894+0000","last_undegraded":"2021-01-15T19:11:29.217894+0000","last_fullsized":"2021-01-15T19:10:46.265441+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:28.840170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:10:28.840170+0000","last_clean_scrub_stamp":"2021-01-15T19:10:28.840170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"14'2","reported_seq":"117","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.226755+0000","last_change":"2021-01-15T19:10:28.206502+0000","last_active":"2021-01-15T19:11:29.226755+0000","last_peered":"2021-01-15T19:11:29.226755+0000","last_clean":"2021-01-15T19:08:47.197231+0000","last_became_active":"2021-01-15T19:10:28.206502+0000","last_became_peered":"2021-01-15T19:10:28.206502+0000","last_unstale":"2021-01-15T19:11:29.226755+0000","last_undegraded":"2021-01-15T19:11:29.226755+0000","last_fullsized":"2021-01-15T19:10:28.203792+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:06.583774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:06.583774+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"181'205","reported_seq":"654","reported_epoch":"201","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:29.220150+0000","last_change":"2021-01-15T19:10:28.205687+0000","last_active":"2021-01-15T19:11:29.220150+0000","last_peered":"2021-01-15T19:11:29.220150+0000","last_clean":"2021-01-15T19:10:26.740865+0000","last_became_active":"2021-01-15T19:10:28.205687+0000","last_became_peered":"2021-01-15T19:10:28.205687+0000","last_unstale":"2021-01-15T19:11:29.220150+0000","last_undegraded":"2021-01-15T19:10:28.199877+0000","last_fullsized":"2021-01-15T19:10:28.199644+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'120","last_scrub_stamp":"2021-01-15T19:09:30.324786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.324786+0000","log_size":205,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"stat_sum":{"num_bytes":16359424,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":63,"num_read_kb":70678,"num_write":82,"num_write_kb":21104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2147483647,7],"acting":[3,2147483647,7],"avail_no_missing":["3(0)","7(2)"],"object_location_counts":[{"shards":"3(0),7(2)","objects":10}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.b","version":"0'0","reported_seq":"207","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:31.194926+0000","last_change":"2021-01-15T19:11:31.195044+0000","last_active":"2021-01-15T19:11:31.194926+0000","last_peered":"2021-01-15T19:11:31.194926+0000","last_clean":"2021-01-15T19:11:31.194926+0000","last_became_active":"2021-01-15T19:08:35.210108+0000","last_became_peered":"2021-01-15T19:08:35.210108+0000","last_unstale":"2021-01-15T19:11:31.194926+0000","last_undegraded":"2021-01-15T19:11:31.194926+0000","last_fullsized":"2021-01-15T19:11:31.194926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:11:31.194925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:11:31.194925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"14'2","reported_seq":"117","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.226045+0000","last_change":"2021-01-15T19:10:28.207318+0000","last_active":"2021-01-15T19:11:29.226045+0000","last_peered":"2021-01-15T19:11:29.226045+0000","last_clean":"2021-01-15T19:08:47.197643+0000","last_became_active":"2021-01-15T19:10:28.207318+0000","last_became_peered":"2021-01-15T19:10:28.207318+0000","last_unstale":"2021-01-15T19:11:29.226045+0000","last_undegraded":"2021-01-15T19:11:29.226045+0000","last_fullsized":"2021-01-15T19:10:28.204163+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:18.403239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:18.403239+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"174'105","reported_seq":"628","reported_epoch":"202","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:30.225612+0000","last_change":"2021-01-15T19:10:46.278531+0000","last_active":"2021-01-15T19:11:30.225612+0000","last_peered":"2021-01-15T19:11:30.225612+0000","last_clean":"2021-01-15T19:10:44.297773+0000","last_became_active":"2021-01-15T19:10:46.278531+0000","last_became_peered":"2021-01-15T19:10:46.278531+0000","last_unstale":"2021-01-15T19:11:30.225612+0000","last_undegraded":"2021-01-15T19:10:46.269078+0000","last_fullsized":"2021-01-15T19:10:46.268881+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"133'46","last_scrub_stamp":"2021-01-15T19:10:03.071240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:03.071240+0000","log_size":105,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":6905856,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":33974,"num_write":24,"num_write_kb":9304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2147483647],"acting":[7,4,2147483647],"avail_no_missing":["7(0)","4(1)"],"object_location_counts":[{"shards":"4(1),7(0)","objects":4}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":"3","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.223733+0000","last_change":"2021-01-15T19:11:30.223733+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:30.223733+0000","last_undegraded":"2021-01-15T19:11:30.223733+0000","last_fullsized":"2021-01-15T19:11:30.223733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:01.966888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:01.966888+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"14'2","reported_seq":"50","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.223120+0000","last_change":"2021-01-15T19:10:27.292093+0000","last_active":"2021-01-15T19:09:47.275830+0000","last_peered":"2021-01-15T19:09:46.273193+0000","last_clean":"2021-01-15T19:08:47.197265+0000","last_became_active":"2021-01-15T19:09:24.206432+0000","last_became_peered":"2021-01-15T19:09:24.206432+0000","last_unstale":"2021-01-15T19:11:29.223120+0000","last_undegraded":"2021-01-15T19:11:29.223120+0000","last_fullsized":"2021-01-15T19:11:29.223120+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:19.037889+0000","last_deep_scrub":"14'2","last_deep_scrub_stamp":"2021-01-15T19:10:19.037889+0000","last_clean_scrub_stamp":"2021-01-15T19:10:19.037889+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"181'402","reported_seq":"1008","reported_epoch":"202","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:30.225274+0000","last_change":"2021-01-15T19:10:46.271726+0000","last_active":"2021-01-15T19:11:30.225274+0000","last_peered":"2021-01-15T19:11:30.225274+0000","last_clean":"2021-01-15T19:10:44.429005+0000","last_became_active":"2021-01-15T19:10:46.271726+0000","last_became_peered":"2021-01-15T19:10:46.271726+0000","last_unstale":"2021-01-15T19:11:30.225274+0000","last_undegraded":"2021-01-15T19:10:46.267718+0000","last_fullsized":"2021-01-15T19:10:46.267503+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'176","last_scrub_stamp":"2021-01-15T19:09:53.888744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:53.888744+0000","log_size":402,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":46006272,"num_objects":24,"num_object_clones":19,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":1,"num_read":101,"num_read_kb":150484,"num_write":178,"num_write_kb":48776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":87048192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,7,2],"acting":[2147483647,7,2],"avail_no_missing":["7(1)","2(2)"],"object_location_counts":[{"shards":"2(2),7(1)","objects":24}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.d","version":"0'0","reported_seq":"3","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.223255+0000","last_change":"2021-01-15T19:11:30.223255+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:30.223255+0000","last_undegraded":"2021-01-15T19:11:30.223255+0000","last_fullsized":"2021-01-15T19:11:30.223255+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:09:25.416133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:09:25.416133+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"14'2","reported_seq":"187","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.224440+0000","last_change":"2021-01-15T19:10:46.279376+0000","last_active":"2021-01-15T19:11:29.224440+0000","last_peered":"2021-01-15T19:11:29.224440+0000","last_clean":"2021-01-15T19:09:22.199740+0000","last_became_active":"2021-01-15T19:10:46.279376+0000","last_became_peered":"2021-01-15T19:10:46.279376+0000","last_unstale":"2021-01-15T19:11:29.224440+0000","last_undegraded":"2021-01-15T19:11:29.224440+0000","last_fullsized":"2021-01-15T19:10:46.273338+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":4,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:02.601833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:02.601833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"170'187","reported_seq":"630","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.222446+0000","last_change":"2021-01-15T19:10:45.265689+0000","last_active":"2021-01-15T19:10:45.265340+0000","last_peered":"2021-01-15T19:10:38.227002+0000","last_clean":"2021-01-15T19:10:26.827342+0000","last_became_active":"2021-01-15T19:10:28.210680+0000","last_became_peered":"2021-01-15T19:10:28.210680+0000","last_unstale":"2021-01-15T19:11:29.222446+0000","last_undegraded":"2021-01-15T19:11:29.222446+0000","last_fullsized":"2021-01-15T19:11:29.222446+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"101'91","last_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_deep_scrub":"101'91","last_deep_scrub_stamp":"2021-01-15T19:09:30.016550+0000","last_clean_scrub_stamp":"2021-01-15T19:09:30.016550+0000","log_size":187,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9297920,"num_objects":10,"num_object_clones":7,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":61,"num_read_kb":25912,"num_write":91,"num_write_kb":11152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,4,2147483647],"acting":[2147483647,4,2147483647],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"1"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.c","version":"0'0","reported_seq":"3","reported_epoch":"202","state":"remapped+peering","last_fresh":"2021-01-15T19:11:30.223252+0000","last_change":"2021-01-15T19:11:30.223252+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:30.223252+0000","last_undegraded":"2021-01-15T19:11:30.223252+0000","last_fullsized":"2021-01-15T19:11:30.223252+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:02.931607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:02.931607+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"0'0","reported_seq":"2","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.224222+0000","last_change":"2021-01-15T19:11:29.224222+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2021-01-15T19:11:29.224222+0000","last_undegraded":"2021-01-15T19:11:29.224222+0000","last_fullsized":"2021-01-15T19:11:29.224222+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:08:53.067906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:08:53.067906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"176'150","reported_seq":"676","reported_epoch":"202","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:30.343794+0000","last_change":"2021-01-15T19:11:30.344167+0000","last_active":"2021-01-15T19:11:30.343794+0000","last_peered":"2021-01-15T19:11:30.343794+0000","last_clean":"2021-01-15T19:10:44.305735+0000","last_became_active":"2021-01-15T19:11:30.234687+0000","last_became_peered":"2021-01-15T19:11:30.234687+0000","last_unstale":"2021-01-15T19:11:30.343794+0000","last_undegraded":"2021-01-15T19:11:30.223516+0000","last_fullsized":"2021-01-15T19:11:30.222222+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'59","last_scrub_stamp":"2021-01-15T19:09:28.771226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:09:28.771226+0000","log_size":150,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":9117696,"num_objects":7,"num_object_clones":5,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":5385,"num_write":71,"num_write_kb":9088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":22585344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,2147483647,5],"acting":[2,2147483647,5],"avail_no_missing":["2(0)","5(2)","7(2)"],"object_location_counts":[{"shards":"2(0),5(2)","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4b","length":"2"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"56","length":"4"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"2"},{"start":"66","length":"1"},{"start":"6d","length":"2"},{"start":"70","length":"2"},{"start":"73","length":"1"},{"start":"78","length":"2"},{"start":"84","length":"1"},{"start":"8d","length":"1"},{"start":"8f","length":"1"}]},{"pgid":"2.f","version":"0'0","reported_seq":"185","reported_epoch":"171","state":"stale+active+undersized","last_fresh":"2021-01-15T19:10:39.210968+0000","last_change":"2021-01-15T19:10:28.207876+0000","last_active":"2021-01-15T19:10:39.210968+0000","last_peered":"2021-01-15T19:10:39.210968+0000","last_clean":"2021-01-15T19:10:26.284995+0000","last_became_active":"2021-01-15T19:10:28.207876+0000","last_became_peered":"2021-01-15T19:10:28.207876+0000","last_unstale":"2021-01-15T19:10:45.259593+0000","last_undegraded":"2021-01-15T19:10:39.210968+0000","last_fullsized":"2021-01-15T19:10:28.204459+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:37.964904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:08:37.964904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"64'3","reported_seq":"224","reported_epoch":"201","state":"active+clean+remapped","last_fresh":"2021-01-15T19:11:29.219669+0000","last_change":"2021-01-15T19:10:40.986858+0000","last_active":"2021-01-15T19:11:29.219669+0000","last_peered":"2021-01-15T19:11:29.219669+0000","last_clean":"2021-01-15T19:11:29.219669+0000","last_became_active":"2021-01-15T19:10:29.205034+0000","last_became_peered":"2021-01-15T19:10:29.205034+0000","last_unstale":"2021-01-15T19:11:29.219669+0000","last_undegraded":"2021-01-15T19:11:29.219669+0000","last_fullsized":"2021-01-15T19:11:29.219669+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"64'3","last_scrub_stamp":"2021-01-15T19:10:40.986734+0000","last_deep_scrub":"64'3","last_deep_scrub_stamp":"2021-01-15T19:08:52.906630+0000","last_clean_scrub_stamp":"2021-01-15T19:10:40.986734+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[{"shards":"2,4","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"172'193","reported_seq":"625","reported_epoch":"201","state":"down","last_fresh":"2021-01-15T19:11:29.224628+0000","last_change":"2021-01-15T19:10:45.269484+0000","last_active":"2021-01-15T19:10:45.265477+0000","last_peered":"2021-01-15T19:10:40.370173+0000","last_clean":"2021-01-15T19:10:27.290581+0000","last_became_active":"2021-01-15T19:10:28.209398+0000","last_became_peered":"2021-01-15T19:10:28.209398+0000","last_unstale":"2021-01-15T19:11:29.224628+0000","last_undegraded":"2021-01-15T19:11:29.224628+0000","last_fullsized":"2021-01-15T19:11:29.224628+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'93","last_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_deep_scrub":"100'93","last_deep_scrub_stamp":"2021-01-15T19:09:29.069708+0000","last_clean_scrub_stamp":"2021-01-15T19:09:29.069708+0000","log_size":193,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12697600,"num_objects":11,"num_object_clones":8,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":22264,"num_write":89,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,2147483647,4],"acting":[2147483647,2147483647,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]},{"pgid":"2.e","version":"0'0","reported_seq":"232","reported_epoch":"202","state":"active+clean","last_fresh":"2021-01-15T19:11:30.232721+0000","last_change":"2021-01-15T19:11:30.232721+0000","last_active":"2021-01-15T19:11:30.232721+0000","last_peered":"2021-01-15T19:11:30.232721+0000","last_clean":"2021-01-15T19:11:30.232721+0000","last_became_active":"2021-01-15T19:11:30.232330+0000","last_became_peered":"2021-01-15T19:11:30.232330+0000","last_unstale":"2021-01-15T19:11:30.232721+0000","last_undegraded":"2021-01-15T19:11:30.232721+0000","last_fullsized":"2021-01-15T19:11:30.232721+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":202,"parent":"0.0","parent_split_bits":4,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:04.241731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:04.241731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.13","version":"14'2","reported_seq":"245","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.223449+0000","last_change":"2021-01-15T19:10:41.214323+0000","last_active":"2021-01-15T19:11:29.223449+0000","last_peered":"2021-01-15T19:11:29.223449+0000","last_clean":"2021-01-15T19:11:29.223449+0000","last_became_active":"2021-01-15T19:10:28.208230+0000","last_became_peered":"2021-01-15T19:10:28.208230+0000","last_unstale":"2021-01-15T19:11:29.223449+0000","last_undegraded":"2021-01-15T19:11:29.223449+0000","last_fullsized":"2021-01-15T19:11:29.223449+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:10:41.214187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:10:41.214187+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"178'119","reported_seq":"533","reported_epoch":"202","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:30.227618+0000","last_change":"2021-01-15T19:10:28.206472+0000","last_active":"2021-01-15T19:11:30.227618+0000","last_peered":"2021-01-15T19:11:30.227618+0000","last_clean":"2021-01-15T19:10:27.071633+0000","last_became_active":"2021-01-15T19:10:28.206472+0000","last_became_peered":"2021-01-15T19:10:28.206472+0000","last_unstale":"2021-01-15T19:11:30.227618+0000","last_undegraded":"2021-01-15T19:10:28.200211+0000","last_fullsized":"2021-01-15T19:10:28.200003+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:08:09.211024+0000","log_size":119,"ondisk_log_size":119,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"stat_sum":{"num_bytes":8413184,"num_objects":7,"num_object_clones":6,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":33,"num_read_kb":54352,"num_write":41,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2686976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,2147483647],"acting":[7,2,2147483647],"avail_no_missing":["7(0)","2(1)"],"object_location_counts":[{"shards":"2(1),7(0)","objects":7}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"},{"start":"79","length":"1"}]},{"pgid":"2.10","version":"0'0","reported_seq":"174","reported_epoch":"201","state":"active+clean","last_fresh":"2021-01-15T19:11:29.225529+0000","last_change":"2021-01-15T19:11:13.224980+0000","last_active":"2021-01-15T19:11:29.225529+0000","last_peered":"2021-01-15T19:11:29.225529+0000","last_clean":"2021-01-15T19:11:29.225529+0000","last_became_active":"2021-01-15T19:08:37.214325+0000","last_became_peered":"2021-01-15T19:08:37.214325+0000","last_unstale":"2021-01-15T19:11:29.225529+0000","last_undegraded":"2021-01-15T19:11:29.225529+0000","last_fullsized":"2021-01-15T19:11:29.225529+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:11:13.224905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:42.075929+0000","last_clean_scrub_stamp":"2021-01-15T19:11:13.224905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"14'2","reported_seq":"35","reported_epoch":"201","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:29.222186+0000","last_change":"2021-01-15T19:10:46.279989+0000","last_active":"2021-01-15T19:11:29.222186+0000","last_peered":"2021-01-15T19:11:29.222186+0000","last_clean":"2021-01-15T19:08:47.197945+0000","last_became_active":"2021-01-15T19:10:46.279989+0000","last_became_peered":"2021-01-15T19:10:46.279989+0000","last_unstale":"2021-01-15T19:11:29.222186+0000","last_undegraded":"2021-01-15T19:10:46.273335+0000","last_fullsized":"2021-01-15T19:10:46.273175+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":5,"last_scrub":"14'2","last_scrub_stamp":"2021-01-15T19:09:16.945387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:48.500993+0000","last_clean_scrub_stamp":"2021-01-15T19:09:16.945387+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":"80","reported_epoch":"201","state":"active+undersized","last_fresh":"2021-01-15T19:11:29.224966+0000","last_change":"2021-01-15T19:10:46.280868+0000","last_active":"2021-01-15T19:11:29.224966+0000","last_peered":"2021-01-15T19:11:29.224966+0000","last_clean":"2021-01-15T19:09:10.329764+0000","last_became_active":"2021-01-15T19:10:46.280868+0000","last_became_peered":"2021-01-15T19:10:46.280868+0000","last_unstale":"2021-01-15T19:11:29.224966+0000","last_undegraded":"2021-01-15T19:11:29.224966+0000","last_fullsized":"2021-01-15T19:10:46.272131+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2021-01-15T19:10:37.898142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:07:55.165623+0000","last_clean_scrub_stamp":"2021-01-15T19:10:37.898142+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"181'206","reported_seq":"60","reported_epoch":"202","state":"active+undersized+degraded","last_fresh":"2021-01-15T19:11:30.445834+0000","last_change":"2021-01-15T19:11:30.446225+0000","last_active":"2021-01-15T19:11:30.445834+0000","last_peered":"2021-01-15T19:11:30.445834+0000","last_clean":"2021-01-15T19:10:26.295820+0000","last_became_active":"2021-01-15T19:11:30.230953+0000","last_became_peered":"2021-01-15T19:11:30.230953+0000","last_unstale":"2021-01-15T19:11:30.445834+0000","last_undegraded":"2021-01-15T19:11:30.223292+0000","last_fullsized":"2021-01-15T19:11:30.223074+0000","mapping_epoch":201,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"146'151","last_scrub_stamp":"2021-01-15T19:10:15.176740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-01-15T19:08:09.211024+0000","last_clean_scrub_stamp":"2021-01-15T19:10:15.176740+0000","log_size":206,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"stat_sum":{"num_bytes":21102592,"num_objects":15,"num_object_clones":11,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":49,"num_read_kb":47272,"num_write":89,"num_write_kb":27312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":44556288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,5,4],"acting":[2147483647,5,4],"avail_no_missing":["5(1)","2(1)","4(2)"],"object_location_counts":[{"shards":"4(2),5(1)","objects":15}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"19","length":"1"},{"start":"26","length":"1"},{"start":"2a","length":"1"},{"start":"2f","length":"1"},{"start":"38","length":"1"},{"start":"3a","length":"1"},{"start":"3e","length":"1"},{"start":"41","length":"1"},{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"4c","length":"1"},{"start":"50","length":"1"},{"start":"53","length":"2"},{"start":"57","length":"3"},{"start":"5c","length":"2"},{"start":"5f","length":"2"},{"start":"62","length":"1"},{"start":"6d","length":"2"},{"start":"71","length":"1"},{"start":"73","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":259129344,"num_objects":170,"num_object_clones":125,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":89,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":9,"num_read":610,"num_read_kb":726319,"num_write":1095,"num_write_kb":267680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":220061696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447791104,"data_stored":775385088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3805,"internal_metadata":0},"log_size":3388,"ondisk_log_size":3388,"up":78,"acting":78,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9184,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":27,"acting":27,"num_store_stats":8},{"poolid":2,"num_pg":18,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":29,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157124,"kb_used_data":102908,"kb_used_omap":0,"kb_used_meta":54208,"kb_avail":94214716,"statfs":{"total":96636764160,"available":96475869184,"internally_reserved":0,"allocated":105377792,"data_stored":199630468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":55508992},"hb_peers":[2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91400000000000003,"15min":0.92000000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1519999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.88600000000000001}]},{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.97499999999999998,"15min":0.97099999999999997},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.341,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91400000000000003,"15min":0.91000000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.002,"15min":0.98999999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2050000000000001,"5min":1.276,"15min":1.276},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.0489999999999999,"15min":1.079},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.379,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1399999999999999,"15min":1.157},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.091}]},{"osd":4,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92400000000000004,"15min":0.96099999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.087,"5min":1.427,"15min":1.427},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.1399999999999999,"15min":1.208},"min":{"1min":0.628,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4710000000000001,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.03}]},{"osd":5,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.155,"15min":1.2010000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8180000000000001,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1040000000000001,"15min":1.139},"min":{"1min":0.71699999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.304,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.133}]},{"osd":6,"last update":"Fri Jan 15 19:11:02 2021","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.014}]}]},{"osd":6,"up_from":0,"seq":566935683092,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84244,"kb_used_data":46476,"kb_used_omap":0,"kb_used_meta":37760,"kb_avail":94287596,"statfs":{"total":96636764160,"available":96550498304,"internally_reserved":0,"allocated":47591424,"data_stored":87650503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":38666240},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2729999999999999,"5min":1.452,"15min":1.452},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0169999999999999,"15min":1.0469999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1819999999999999,"5min":4.1269999999999998,"15min":4.1269999999999998},"last":1.169}]},{"osd":2,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.83599999999999997,"15min":0.85499999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.008,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.798},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.085,"15min":1.0920000000000001},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.3480000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Jan 15 19:09:55 2021","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1479999999999999,"15min":1.1719999999999999},"min":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4910000000000001,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.494},{"interface":"front","average":{"1min":1.111,"5min":1.427,"15min":1.4790000000000001},"min":{"1min":0.70199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6519999999999999,"5min":10.584,"15min":10.584},"last":1.0580000000000001}]},{"osd":4,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.3620000000000001,"15min":1.3879999999999999},"min":{"1min":0.755,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1699999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.3169999999999999,"15min":1.337},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.5469999999999999,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.018},{"interface":"front","average":{"1min":1.228,"5min":1.391,"15min":1.4179999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.673,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.3580000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:08:51 2021","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.21},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.276}]},{"osd":7,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.488,"15min":1.512},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.7629999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.343},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.587,"15min":1.621},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0150000000000001,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.6499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94444,"kb_used_data":54436,"kb_used_omap":0,"kb_used_meta":40000,"kb_avail":94277396,"statfs":{"total":96636764160,"available":96540053504,"internally_reserved":0,"allocated":55742464,"data_stored":82736452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40960000},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.048,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83499999999999996,"15min":0.83899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.94199999999999995,"15min":0.95299999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.97499999999999998,"15min":0.99099999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3260000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.90300000000000002,"15min":0.92300000000000004},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0549999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.181,"15min":1.198},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.506,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5369999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.95099999999999996}]},{"osd":5,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.681,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.141},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.3580000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.724,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Jan 15 19:08:53 2021","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.236},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.4219999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:09:56 2021","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.4350000000000001,"15min":1.4550000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.6819999999999999,"5min":1.96,"15min":1.96},"last":1.22},{"interface":"front","average":{"1min":1.367,"5min":1.5349999999999999,"15min":1.5629999999999999},"min":{"1min":1.024,"5min":1.024,"15min":1.024},"max":{"1min":1.847,"5min":2.024,"15min":2.024},"last":1.345}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135644,"kb_used_data":82900,"kb_used_omap":0,"kb_used_meta":52736,"kb_avail":94236196,"statfs":{"total":96636764160,"available":96497864704,"internally_reserved":0,"allocated":84889600,"data_stored":153526882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54001664},"hb_peers":[3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.54700000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2629999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jan 15 19:09:52 2021","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95699999999999996,"15min":0.94899999999999995},"min":{"1min":0.76900000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.83099999999999996,"15min":0.81499999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.036,"5min":1.0780000000000001,"15min":1.0860000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.84299999999999997}]},{"osd":4,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.1220000000000001,"15min":1.1319999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4550000000000001,"5min":1.77,"15min":1.77},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.218,"15min":1.244},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7230000000000001,"5min":1.859,"15min":1.859},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.252,"15min":1.27},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6419999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1879999999999999,"15min":1.2050000000000001},"min":{"1min":0.83499999999999996,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.94499999999999995}]},{"osd":6,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.80500000000000005}]},{"osd":7,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.353,"15min":1.3520000000000001},"min":{"1min":0.98399999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.8,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.095},{"interface":"front","average":{"1min":1.363,"5min":1.393,"15min":1.395},"min":{"1min":1.069,"5min":1.014,"15min":1.014},"max":{"1min":1.952,"5min":2.133,"15min":2.133},"last":1.008}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":13736,"kb_used_omap":0,"kb_used_meta":19328,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":14065664,"data_stored":21855876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":19791872},"hb_peers":[2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.755},{"interface":"front","average":{"1min":0.87,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0629999999999999,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Jan 15 19:09:54 2021","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.56100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.052,"5min":1.0029999999999999,"15min":0.995},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.99199999999999999,"15min":1.026},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":4.4370000000000003,"5min":4.4370000000000003,"15min":4.4370000000000003},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.151,"15min":1.1419999999999999},"min":{"1min":0.71199999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.258,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3080000000000001,"15min":1.3100000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.6030000000000002,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.4430000000000001,"15min":1.4730000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.9349999999999996,"5min":4.9349999999999996,"15min":4.9349999999999996},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.9900000000000002,"5min":2.9900000000000002,"15min":2.9900000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":0.98599999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:10:55 2021","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":0.93300000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":3.137,"5min":3.137,"15min":3.137},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.4219999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.89000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129852,"kb_used_data":77876,"kb_used_omap":0,"kb_used_meta":51968,"kb_avail":94241988,"statfs":{"total":96636764160,"available":96503795712,"internally_reserved":0,"allocated":79745024,"data_stored":154328708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53215232},"hb_peers":[2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.875,"15min":0.88500000000000001},"min":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3280000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.88200000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.252,"5min":1.409,"15min":1.409},"last":1.276}]},{"osd":1,"last update":"Fri Jan 15 19:09:51 2021","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.91400000000000003,"15min":0.90600000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.617,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2050000000000001,"15min":1.1990000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.429}]},{"osd":2,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98099999999999998,"15min":0.99299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8300000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.377},"min":{"1min":0.72399999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.177,"15min":1.1930000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3479999999999999,"5min":2.758,"15min":2.758},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.123,"15min":1.1339999999999999},"min":{"1min":0.77100000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7170000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.99299999999999999,"15min":1.0549999999999999},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3180000000000001,"5min":1.571,"15min":1.571},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.373,"5min":1.304,"15min":1.333},"min":{"1min":0.88900000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.637,"5min":2.637,"15min":2.637},"last":1.361}]},{"osd":6,"last update":"Fri Jan 15 19:11:04 2021","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.996}]},{"osd":7,"last update":"Fri Jan 15 19:10:52 2021","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.476,"15min":1.5429999999999999},"min":{"1min":0.876,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":1.04},{"interface":"front","average":{"1min":1.248,"5min":1.333,"15min":1.381},"min":{"1min":0.76900000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7310000000000001,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.1060000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":19352,"kb_used_omap":0,"kb_used_meta":13248,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":19816448,"data_stored":18812548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13565952},"hb_peers":[2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jan 15 19:09:53 2021","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.95699999999999996,"15min":0.96999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.363,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.157},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4990000000000001,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.038}]},{"osd":1,"last update":"Fri Jan 15 19:09:53 2021","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.998,"15min":0.98899999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.181,"15min":1.173},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.3500000000000001}]},{"osd":2,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.94399999999999995,"15min":0.90600000000000003},"min":{"1min":0.64600000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.0369999999999999,"5min":6.3600000000000003,"15min":6.3600000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0229999999999999,"15min":1.0609999999999999},"min":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.2809999999999999,"5min":1.601,"15min":1.601},"last":1.1080000000000001}]},{"osd":3,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.044,"15min":1.052},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.595,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.748},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.238,"15min":1.23},"min":{"1min":0.78200000000000003,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1299999999999999,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.90800000000000003}]},{"osd":4,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.0409999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.6299999999999999,"5min":2.298,"15min":2.298},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.272,"15min":1.2889999999999999},"min":{"1min":0.79200000000000004,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.218,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Jan 15 19:11:00 2021","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.0669999999999999}]},{"osd":7,"last update":"Fri Jan 15 19:10:54 2021","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.367,"15min":1.4059999999999999},"min":{"1min":0.97199999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0049999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.155},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.46,"15min":1.468},"min":{"1min":1.0589999999999999,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.3050000000000002,"5min":6.7480000000000002,"15min":6.7480000000000002},"last":1.2450000000000001}]}]}]}} 2021-01-15T19:11:34.580 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2021-01-15T19:11:34.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set rbd pg_num 28 2021-01-15T19:11:34.655 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:34.654+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:34.671 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:34.671 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:34.681 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:34.718 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:34.716+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:34.805 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:34.806 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:34.815 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:34.819 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:34.817+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:34.920 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:34.918+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:34.940 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:34.941 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:34.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:35.034 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:35.033+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:35.115 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.115 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:35.120 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:35.120+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:35.124 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:35.221 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:35.220+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 2 pg_num to 28 2021-01-15T19:11:35.291 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.291 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:35.300 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:35.322 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:35.321+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:35.423 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:35.422+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:35.425 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.425 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:35.434 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:35.523 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:35.523+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:35.570 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.571 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:35.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:35.624 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:35.623+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:35.711 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.712 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:35.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:35.726 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:35.725+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:35.826 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:35.826+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:35.852 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.852 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:35.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:35.928 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:35.927+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:35.993 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:35.994 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:36.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:36.028 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:36.027+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:36.129 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:36.128+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:36.137 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:36.138 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:36.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:36.230 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:36.229+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:36.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2021-01-15T19:11:36.284 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:36.285 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:36.286 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:36.295 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:36.340 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:36.339+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:36.429 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:36.430 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:36.432 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:36.431+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:36.439 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:36.448 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:36.468 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:36.488 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:36.509 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:36.529 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:36.533 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:36.532+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:36.550 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:36.550 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:36.551 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:36.551 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:36.551 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:36.552 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:36.552 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:36.552 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:36.552 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:36.553 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:36.553 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:36.554 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:36.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:36.605 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:36.605 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:36.606 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:36.606 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:36.606 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:36.615 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:36.634 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:36.633+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:36.735 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:36.734+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:36.781 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:36.782 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:36.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:36.782 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:36.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.783 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:825 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:36.783 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:36.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 148.59548339400001, 2021-01-15T19:11:36.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999, 2021-01-15T19:11:36.784 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.784 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:36.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.785 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 825 2021-01-15T19:11:36.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.785 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:36.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183861+0000", 2021-01-15T19:11:36.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0688999999999999e-05 2021-01-15T19:11:36.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183942+0000", 2021-01-15T19:11:36.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4549000000000001e-05 2021-01-15T19:11:36.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183956+0000", 2021-01-15T19:11:36.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3732e-05 2021-01-15T19:11:36.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183970+0000", 2021-01-15T19:11:36.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4869999999999999e-06 2021-01-15T19:11:36.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.183976+0000", 2021-01-15T19:11:36.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013431070000000001 2021-01-15T19:11:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185320+0000", 2021-01-15T19:11:36.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012978499999999999 2021-01-15T19:11:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185449+0000", 2021-01-15T19:11:36.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024224664 2021-01-15T19:11:36.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.209674+0000", 2021-01-15T19:11:36.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025813012999999999 2021-01-15T19:11:36.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.795 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.796 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:826 3.7s0 3:e0e4bbc9:::smithi00140323-39:26 [read 0~4710400,checksum] snapc 0=[] ondisk+read+known_if_redirected e80)", 2021-01-15T19:11:36.796 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:36.797 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 148.595320453, 2021-01-15T19:11:36.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001, 2021-01-15T19:11:36.797 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.797 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:36.798 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.798 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.798 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.798 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 826 2021-01-15T19:11:36.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.799 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:36.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184024+0000", 2021-01-15T19:11:36.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7039000000000001e-05 2021-01-15T19:11:36.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184081+0000", 2021-01-15T19:11:36.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7349999999999999e-06 2021-01-15T19:11:36.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184088+0000", 2021-01-15T19:11:36.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2021-01-15T19:11:36.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184100+0000", 2021-01-15T19:11:36.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1329999999999998e-06 2021-01-15T19:11:36.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.184105+0000", 2021-01-15T19:11:36.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675026 2021-01-15T19:11:36.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185780+0000", 2021-01-15T19:11:36.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013533299999999999 2021-01-15T19:11:36.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.185915+0000", 2021-01-15T19:11:36.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027900017999999999 2021-01-15T19:11:36.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:08.213815+0000", 2021-01-15T19:11:36.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.029791344000000001 2021-01-15T19:11:36.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.810 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.811 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:861 3.ds1 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:36.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:36.812 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.434277807, 2021-01-15T19:11:36.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999, 2021-01-15T19:11:36.812 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:36.812 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.813 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 861 2021-01-15T19:11:36.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.813 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:36.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345067+0000", 2021-01-15T19:11:36.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0457999999999997e-05 2021-01-15T19:11:36.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345117+0000", 2021-01-15T19:11:36.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9490000000000001e-06 2021-01-15T19:11:36.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345123+0000", 2021-01-15T19:11:36.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2109e-05 2021-01-15T19:11:36.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345135+0000", 2021-01-15T19:11:36.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9820000000000008e-06 2021-01-15T19:11:36.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345143+0000", 2021-01-15T19:11:36.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0621e-05 2021-01-15T19:11:36.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345214+0000", 2021-01-15T19:11:36.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4740000000000001e-06 2021-01-15T19:11:36.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345217+0000", 2021-01-15T19:11:36.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0021740210000001 2021-01-15T19:11:36.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347391+0000", 2021-01-15T19:11:36.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028383400000000001 2021-01-15T19:11:36.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.347675+0000", 2021-01-15T19:11:36.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017923129999999999 2021-01-15T19:11:36.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:36.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.365598+0000", 2021-01-15T19:11:36.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055192180000000002 2021-01-15T19:11:36.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:36.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371117+0000", 2021-01-15T19:11:36.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038388700000000002 2021-01-15T19:11:36.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:36.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371501+0000", 2021-01-15T19:11:36.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037828699999999999 2021-01-15T19:11:36.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.371880+0000", 2021-01-15T19:11:36.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0268129699999999 2021-01-15T19:11:36.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.841 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.842 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.843 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:862 3.ds1 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:36.843 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:36.844 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.43417035100001, 2021-01-15T19:11:36.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993, 2021-01-15T19:11:36.845 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.845 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:36.846 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.846 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.847 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.847 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 862 2021-01-15T19:11:36.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.848 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:36.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345174+0000", 2021-01-15T19:11:36.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1087e-05 2021-01-15T19:11:36.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345195+0000", 2021-01-15T19:11:36.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6500000000000004e-06 2021-01-15T19:11:36.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345200+0000", 2021-01-15T19:11:36.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0350000000000001e-06 2021-01-15T19:11:36.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345208+0000", 2021-01-15T19:11:36.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5870000000000001e-06 2021-01-15T19:11:36.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345211+0000", 2021-01-15T19:11:36.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010846 2021-01-15T19:11:36.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345320+0000", 2021-01-15T19:11:36.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7020000000000001e-06 2021-01-15T19:11:36.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.345323+0000", 2021-01-15T19:11:36.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.002912295 2021-01-15T19:11:36.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348235+0000", 2021-01-15T19:11:36.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4149999999999998e-05 2021-01-15T19:11:36.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:36.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348329+0000", 2021-01-15T19:11:36.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019050414000000002 2021-01-15T19:11:36.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367379+0000", 2021-01-15T19:11:36.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020203800000000001 2021-01-15T19:11:36.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:36.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.367581+0000", 2021-01-15T19:11:36.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048699879999999996 2021-01-15T19:11:36.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372451+0000", 2021-01-15T19:11:36.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018168599999999999 2021-01-15T19:11:36.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372633+0000", 2021-01-15T19:11:36.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033490100000000002 2021-01-15T19:11:36.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.372968+0000", 2021-01-15T19:11:36.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.027793993 2021-01-15T19:11:36.867 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.868 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.868 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:875 3.ds1 3:bd2d5ff7:::smithi00140323-2:head [create,setxattr ntQVt[arV\\sZ (12980) in=12b,setxattr cbiTQrXTtN[rcsXX]e`cSaeckYftjjZYtT\\Q^igZ (947) in=40b,setxattr Ncc_QolebZhUSbdoam]TtS[WPO`dcS`[mQ]`jeUq (13680) in=40b,setxattr aaW^bm`niOo[qYndkoRPSTsgTRVZsjTbdqNTWSbq (139) in=40b,setxattr ^mjiNm[Oc`m`tdgmetjcaYsTRWfVX[Y]OrieZYts (4560) in=40b,setxattr a_tYdUeqQ\\TlaV]SnRrNNZnbQiX\\r^ocf\\YnUjhY (15364) in=40b,setxattr mQh]P]enrma\\V`gdWlZagTV^ (12980) in=24b,setxattr tOhOYXahpesOiNUhkPgPbV`fUejZib__YUhaXTTR (1051) in=40b,setxattr ZZc]SnhtrtmognZg[mgTlar`PsaWsenZek`Qf`eY (1049) in=40b,setxattr hUURNtkOoXqXi]eatXq^SUacfR`Q]k[iiNgbi` (12980) in=38b,setxattr aQVfqQ[ifabWts (12980) in=14b,setxattr tY]sPTr\\QlZbSqcPigYORZQpfels`mkh^Pkner\\] (805) in=40b,setxattr \\RsX`is\\W_nOa_UaifQU[aaoOZRre^^RegbbOcqT (1891) in=40b,setxattr nqeolZZpfWUXqprf]\\PoS]lk_pef`^QpSreiVttQ (245) in=40b,setxattr \\OeXfOVSON\\t^bQb[lWXiXaYmVQXUcSqQnNh`j`t (4768) in=40b,setxattr eo^U[lhQUeVQ_Nc_bNdcfmXp_UQ`WXl_is_NkOfS (1577) in=40b,setxattr XUbk]peZmr_iot^gmY`pgm`eOnTpNT_SQd`aVj[e (1437) in=40b,setxattr XkXTUfbiTjaQaf\\fq[_\\antZOOTes[[ZYantUYqi (4560) in=40b,setxattr ab]YiQ (12980) in=6b,setxattr ZaSYU^RRVdSq_eaQfajUXsi\\ah^nfW_shiX]UlSd (14172) in=40b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:36.869 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:36.869 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.301103485, 2021-01-15T19:11:36.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995, 2021-01-15T19:11:36.869 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.869 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:36.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.870 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.870 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 875 2021-01-15T19:11:36.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:36.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478241+0000", 2021-01-15T19:11:36.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019290299999999999 2021-01-15T19:11:36.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478434+0000", 2021-01-15T19:11:36.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2449999999999998e-06 2021-01-15T19:11:36.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478442+0000", 2021-01-15T19:11:36.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3258000000000001e-05 2021-01-15T19:11:36.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478455+0000", 2021-01-15T19:11:36.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6000000000000001e-06 2021-01-15T19:11:36.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478463+0000", 2021-01-15T19:11:36.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7334999999999999e-05 2021-01-15T19:11:36.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478520+0000", 2021-01-15T19:11:36.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0210000000000001e-06 2021-01-15T19:11:36.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:12.478524+0000", 2021-01-15T19:11:36.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.86991831100000006 2021-01-15T19:11:36.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348443+0000", 2021-01-15T19:11:36.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029576899999999998 2021-01-15T19:11:36.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.348738+0000", 2021-01-15T19:11:36.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002844652 2021-01-15T19:11:36.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:36.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.351583+0000", 2021-01-15T19:11:36.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075896210000000004 2021-01-15T19:11:36.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:36.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.359173+0000", 2021-01-15T19:11:36.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095944699999999999 2021-01-15T19:11:36.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:36.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360132+0000", 2021-01-15T19:11:36.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031317299999999998 2021-01-15T19:11:36.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:13.360445+0000", 2021-01-15T19:11:36.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.88220433499999995 2021-01-15T19:11:36.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.886 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:36.886 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:36.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 139.40513580199999, 2021-01-15T19:11:36.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999, 2021-01-15T19:11:36.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.887 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:36.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.887 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:36.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.888 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:36.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374209+0000", 2021-01-15T19:11:36.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6881e-05 2021-01-15T19:11:36.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374265+0000", 2021-01-15T19:11:36.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1969999999999999e-06 2021-01-15T19:11:36.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374273+0000", 2021-01-15T19:11:36.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4053000000000001e-05 2021-01-15T19:11:36.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374287+0000", 2021-01-15T19:11:36.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7589999999999997e-06 2021-01-15T19:11:36.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374291+0000", 2021-01-15T19:11:36.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7015999999999998e-05 2021-01-15T19:11:36.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:36.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4030000000000001e-06 2021-01-15T19:11:36.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374341+0000", 2021-01-15T19:11:36.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82280367099999996 2021-01-15T19:11:36.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197145+0000", 2021-01-15T19:11:36.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3752e-05 2021-01-15T19:11:36.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.197158+0000", 2021-01-15T19:11:36.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82294973199999999 2021-01-15T19:11:36.898 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.898 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.898 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.899 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:36.899 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:36.899 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 139.40500567199999, 2021-01-15T19:11:36.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002, 2021-01-15T19:11:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:36.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374339+0000", 2021-01-15T19:11:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8108000000000002e-05 2021-01-15T19:11:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374377+0000", 2021-01-15T19:11:36.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2420000000000002e-06 2021-01-15T19:11:36.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374381+0000", 2021-01-15T19:11:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3529999999999997e-06 2021-01-15T19:11:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374388+0000", 2021-01-15T19:11:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.304e-06 2021-01-15T19:11:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374392+0000", 2021-01-15T19:11:36.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:36.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374427+0000", 2021-01-15T19:11:36.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2021-01-15T19:11:36.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374429+0000", 2021-01-15T19:11:36.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82401203099999998 2021-01-15T19:11:36.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198441+0000", 2021-01-15T19:11:36.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2883e-05 2021-01-15T19:11:36.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198454+0000", 2021-01-15T19:11:36.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82411476900000002 2021-01-15T19:11:36.911 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.911 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.911 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.912 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s0 3.4220fe00 (undecoded) ondisk+read+known_if_redirected e89)", 2021-01-15T19:11:36.912 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:36.912 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 139.40492026300001, 2021-01-15T19:11:36.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001, 2021-01-15T19:11:36.913 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.913 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:36.913 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.913 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.913 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.914 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:36.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.914 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:36.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374424+0000", 2021-01-15T19:11:36.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9403e-05 2021-01-15T19:11:36.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374454+0000", 2021-01-15T19:11:36.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6820000000000001e-06 2021-01-15T19:11:36.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374457+0000", 2021-01-15T19:11:36.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5699999999999998e-06 2021-01-15T19:11:36.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374464+0000", 2021-01-15T19:11:36.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5719999999999999e-06 2021-01-15T19:11:36.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374467+0000", 2021-01-15T19:11:36.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9856e-05 2021-01-15T19:11:36.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374497+0000", 2021-01-15T19:11:36.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5199999999999995e-07 2021-01-15T19:11:36.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:17.374498+0000", 2021-01-15T19:11:36.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82397995000000002 2021-01-15T19:11:36.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198478+0000", 2021-01-15T19:11:36.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0233e-05 2021-01-15T19:11:36.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.198488+0000", 2021-01-15T19:11:36.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82406411800000001 2021-01-15T19:11:36.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.925 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.925 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:974 3.18s0 3:1c18e8c7:::smithi00140323-46:head [rollback 2a] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:36.926 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:36.926 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 133.57374587499999, 2021-01-15T19:11:36.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767, 2021-01-15T19:11:36.926 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.926 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:36.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.927 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.927 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 974 2021-01-15T19:11:36.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.928 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:36.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205599+0000", 2021-01-15T19:11:36.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2951000000000001e-05 2021-01-15T19:11:36.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205621+0000", 2021-01-15T19:11:36.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.618e-06 2021-01-15T19:11:36.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205628+0000", 2021-01-15T19:11:36.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0541e-05 2021-01-15T19:11:36.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205639+0000", 2021-01-15T19:11:36.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6019999999999996e-06 2021-01-15T19:11:36.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205645+0000", 2021-01-15T19:11:36.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7037999999999998e-05 2021-01-15T19:11:36.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205702+0000", 2021-01-15T19:11:36.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1969999999999996e-06 2021-01-15T19:11:36.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:36.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205710+0000", 2021-01-15T19:11:36.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051931640000000001 2021-01-15T19:11:36.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210904+0000", 2021-01-15T19:11:36.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8159999999999998e-06 2021-01-15T19:11:36.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210906+0000", 2021-01-15T19:11:36.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0112695279999999 2021-01-15T19:11:36.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222176+0000", 2021-01-15T19:11:36.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021074000000000001 2021-01-15T19:11:36.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.222387+0000", 2021-01-15T19:11:36.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011262939999999999 2021-01-15T19:11:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.223513+0000", 2021-01-15T19:11:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055524529999999997 2021-01-15T19:11:36.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:36.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229065+0000", 2021-01-15T19:11:36.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031451899999999997 2021-01-15T19:11:36.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:36.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229380+0000", 2021-01-15T19:11:36.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000435306 2021-01-15T19:11:36.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229815+0000", 2021-01-15T19:11:36.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.024216767 2021-01-15T19:11:36.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.944 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.945 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:975 3.18s0 3:1c18e8c7:::smithi00140323-46:head [append 0~0] snapc 44=[44,43,41,3e,3a,38,2f,2a,26,1b,19] ondisk+write+known_if_redirected e95)", 2021-01-15T19:11:36.945 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:36.945 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 133.57365722599999, 2021-01-15T19:11:36.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999, 2021-01-15T19:11:36.945 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:36.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.946 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.946 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 975 2021-01-15T19:11:36.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.947 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:36.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205687+0000", 2021-01-15T19:11:36.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3671000000000001e-05 2021-01-15T19:11:36.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205711+0000", 2021-01-15T19:11:36.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0770000000000001e-06 2021-01-15T19:11:36.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205715+0000", 2021-01-15T19:11:36.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2030000000000003e-06 2021-01-15T19:11:36.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205722+0000", 2021-01-15T19:11:36.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2739999999999999e-06 2021-01-15T19:11:36.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205725+0000", 2021-01-15T19:11:36.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1014000000000002e-05 2021-01-15T19:11:36.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205786+0000", 2021-01-15T19:11:36.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1479e-05 2021-01-15T19:11:36.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:36.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.205798+0000", 2021-01-15T19:11:36.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051468240000000004 2021-01-15T19:11:36.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210945+0000", 2021-01-15T19:11:36.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2021-01-15T19:11:36.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:36.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:23.210948+0000", 2021-01-15T19:11:36.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.018903444 2021-01-15T19:11:36.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229851+0000", 2021-01-15T19:11:36.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109078 2021-01-15T19:11:36.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.229960+0000", 2021-01-15T19:11:36.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033860699999999999 2021-01-15T19:11:36.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:36.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.230299+0000", 2021-01-15T19:11:36.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010128959999999999 2021-01-15T19:11:36.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:36.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.231312+0000", 2021-01-15T19:11:36.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038435299999999999 2021-01-15T19:11:36.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:36.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235155+0000", 2021-01-15T19:11:36.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000184649 2021-01-15T19:11:36.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.235340+0000", 2021-01-15T19:11:36.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0296525759999999 2021-01-15T19:11:36.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.964 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1055 3.8s0 3:122eef37:::smithi00140323-28:head [copy-from ver 49] snapc 4c=[4c,4b,44,41,3a,26,19] ondisk+write+known_if_redirected e105)", 2021-01-15T19:11:36.964 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:36.964 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 123.590149168, 2021-01-15T19:11:36.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003, 2021-01-15T19:11:36.964 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.965 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:36.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.965 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1055 2021-01-15T19:11:36.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.966 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:36.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189195+0000", 2021-01-15T19:11:36.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3006e-05 2021-01-15T19:11:36.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189248+0000", 2021-01-15T19:11:36.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0099999999999995e-06 2021-01-15T19:11:36.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189256+0000", 2021-01-15T19:11:36.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4601e-05 2021-01-15T19:11:36.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189271+0000", 2021-01-15T19:11:36.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7719999999999994e-06 2021-01-15T19:11:36.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189279+0000", 2021-01-15T19:11:36.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2069999999999994e-05 2021-01-15T19:11:36.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189341+0000", 2021-01-15T19:11:36.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2806000000000002e-05 2021-01-15T19:11:36.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:36.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189364+0000", 2021-01-15T19:11:36.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048370790000000002 2021-01-15T19:11:36.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194201+0000", 2021-01-15T19:11:36.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014282800000000001 2021-01-15T19:11:36.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194343+0000", 2021-01-15T19:11:36.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024624099999999999 2021-01-15T19:11:36.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:36.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.218968+0000", 2021-01-15T19:11:36.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014584811 2021-01-15T19:11:36.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:36.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233552+0000", 2021-01-15T19:11:36.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5065e-05 2021-01-15T19:11:36.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:36.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233627+0000", 2021-01-15T19:11:36.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9772999999999997e-05 2021-01-15T19:11:36.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233707+0000", 2021-01-15T19:11:36.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044511921000000003 2021-01-15T19:11:36.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.980 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.981 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1056 3.8s0 3:122eef37:::smithi00140323-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e105)", 2021-01-15T19:11:36.981 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:36.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 123.590017093, 2021-01-15T19:11:36.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997, 2021-01-15T19:11:36.982 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:36.982 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:36.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:36.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:36.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:36.983 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1056 2021-01-15T19:11:36.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.983 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:36.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:36.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:36.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:36.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:36.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189327+0000", 2021-01-15T19:11:36.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8340000000000002e-05 2021-01-15T19:11:36.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:36.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189366+0000", 2021-01-15T19:11:36.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:36.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:36.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189372+0000", 2021-01-15T19:11:36.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2948e-05 2021-01-15T19:11:36.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:36.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189385+0000", 2021-01-15T19:11:36.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4829999999999999e-06 2021-01-15T19:11:36.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:36.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189391+0000", 2021-01-15T19:11:36.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0112999999999999e-05 2021-01-15T19:11:36.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189441+0000", 2021-01-15T19:11:36.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4530000000000008e-06 2021-01-15T19:11:36.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:36.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.189450+0000", 2021-01-15T19:11:36.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050672540000000002 2021-01-15T19:11:36.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194518+0000", 2021-01-15T19:11:36.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6997999999999999e-05 2021-01-15T19:11:36.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:36.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.194575+0000", 2021-01-15T19:11:36.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036027545000000001 2021-01-15T19:11:36.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230602+0000", 2021-01-15T19:11:36.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014945799999999999 2021-01-15T19:11:36.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:36.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.230752+0000", 2021-01-15T19:11:36.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029894990000000001 2021-01-15T19:11:36.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:36.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233741+0000", 2021-01-15T19:11:36.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8907999999999996e-05 2021-01-15T19:11:36.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:36.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233830+0000", 2021-01-15T19:11:36.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8619e-05 2021-01-15T19:11:36.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:36.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:33.233919+0000", 2021-01-15T19:11:36.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044591359999999997 2021-01-15T19:11:36.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.998 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:36.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:36.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:36.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:36.999 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1096 3.8s0 3:146671cc:::smithi00140323-43:head [copy-from ver 34] snapc 53=[53,50,4b,44,41,26] ondisk+write+known_if_redirected e110)", 2021-01-15T19:11:37.000 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:37.000 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 118.586881576, 2021-01-15T19:11:37.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997, 2021-01-15T19:11:37.000 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.001 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.001 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.002 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1096 2021-01-15T19:11:37.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.002 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:37.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192463+0000", 2021-01-15T19:11:37.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9348000000000001e-05 2021-01-15T19:11:37.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192492+0000", 2021-01-15T19:11:37.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.049e-06 2021-01-15T19:11:37.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192498+0000", 2021-01-15T19:11:37.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2298e-05 2021-01-15T19:11:37.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192511+0000", 2021-01-15T19:11:37.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2479999999999999e-06 2021-01-15T19:11:37.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192518+0000", 2021-01-15T19:11:37.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3615000000000001e-05 2021-01-15T19:11:37.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192571+0000", 2021-01-15T19:11:37.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3696999999999999e-05 2021-01-15T19:11:37.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192585+0000", 2021-01-15T19:11:37.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029023510000000001 2021-01-15T19:11:37.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195487+0000", 2021-01-15T19:11:37.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014890899999999999 2021-01-15T19:11:37.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195636+0000", 2021-01-15T19:11:37.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021567020999999999 2021-01-15T19:11:37.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.217203+0000", 2021-01-15T19:11:37.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013486748999999999 2021-01-15T19:11:37.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230690+0000", 2021-01-15T19:11:37.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031812139999999999 2021-01-15T19:11:37.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233871+0000", 2021-01-15T19:11:37.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4971999999999999e-05 2021-01-15T19:11:37.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.233956+0000", 2021-01-15T19:11:37.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041493470999999997 2021-01-15T19:11:37.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1097 3.8s0 3:146671cc:::smithi00140323-43:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e110)", 2021-01-15T19:11:37.021 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:37.021 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 118.58677754999999, 2021-01-15T19:11:37.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998, 2021-01-15T19:11:37.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.022 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.022 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.023 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1097 2021-01-15T19:11:37.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.023 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:37.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192567+0000", 2021-01-15T19:11:37.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9929999999999999e-05 2021-01-15T19:11:37.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192607+0000", 2021-01-15T19:11:37.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:37.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192616+0000", 2021-01-15T19:11:37.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2021-01-15T19:11:37.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192630+0000", 2021-01-15T19:11:37.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8019999999999999e-06 2021-01-15T19:11:37.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192635+0000", 2021-01-15T19:11:37.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5670999999999998e-05 2021-01-15T19:11:37.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192670+0000", 2021-01-15T19:11:37.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0151e-05 2021-01-15T19:11:37.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:37.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.192680+0000", 2021-01-15T19:11:37.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031563049999999999 2021-01-15T19:11:37.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195837+0000", 2021-01-15T19:11:37.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9164000000000004e-05 2021-01-15T19:11:37.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:37.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.195906+0000", 2021-01-15T19:11:37.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034234061000000003 2021-01-15T19:11:37.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230140+0000", 2021-01-15T19:11:37.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012296 2021-01-15T19:11:37.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:37.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.230263+0000", 2021-01-15T19:11:37.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0075329419999999999 2021-01-15T19:11:37.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237796+0000", 2021-01-15T19:11:37.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017253700000000001 2021-01-15T19:11:37.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.237968+0000", 2021-01-15T19:11:37.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015192800000000001 2021-01-15T19:11:37.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:38.238120+0000", 2021-01-15T19:11:37.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045553496999999998 2021-01-15T19:11:37.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.042 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1290 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:37.043 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:37.043 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.575790800999997, 2021-01-15T19:11:37.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999, 2021-01-15T19:11:37.044 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.044 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.044 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.045 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.045 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.045 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1290 2021-01-15T19:11:37.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.045 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:37.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203554+0000", 2021-01-15T19:11:37.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9006999999999999e-05 2021-01-15T19:11:37.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203583+0000", 2021-01-15T19:11:37.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9860000000000002e-06 2021-01-15T19:11:37.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203588+0000", 2021-01-15T19:11:37.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9539999999999993e-06 2021-01-15T19:11:37.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203597+0000", 2021-01-15T19:11:37.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000003e-06 2021-01-15T19:11:37.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203601+0000", 2021-01-15T19:11:37.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7083999999999998e-05 2021-01-15T19:11:37.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203658+0000", 2021-01-15T19:11:37.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2663000000000001e-05 2021-01-15T19:11:37.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203670+0000", 2021-01-15T19:11:37.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003243508 2021-01-15T19:11:37.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.206914+0000", 2021-01-15T19:11:37.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100314 2021-01-15T19:11:37.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207014+0000", 2021-01-15T19:11:37.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027748048000000001 2021-01-15T19:11:37.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.234762+0000", 2021-01-15T19:11:37.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031208693999999999 2021-01-15T19:11:37.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.059 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.060 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1291 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:37.061 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:37.061 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.575695280000005, 2021-01-15T19:11:37.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999, 2021-01-15T19:11:37.061 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.062 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.062 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.063 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1291 2021-01-15T19:11:37.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.063 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:37.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203649+0000", 2021-01-15T19:11:37.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9264000000000001e-05 2021-01-15T19:11:37.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203698+0000", 2021-01-15T19:11:37.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2430000000000002e-06 2021-01-15T19:11:37.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203705+0000", 2021-01-15T19:11:37.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.131e-05 2021-01-15T19:11:37.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203716+0000", 2021-01-15T19:11:37.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4909999999999999e-06 2021-01-15T19:11:37.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203720+0000", 2021-01-15T19:11:37.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000117964 2021-01-15T19:11:37.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203838+0000", 2021-01-15T19:11:37.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2325e-05 2021-01-15T19:11:37.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:37.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203851+0000", 2021-01-15T19:11:37.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035549399999999999 2021-01-15T19:11:37.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207406+0000", 2021-01-15T19:11:37.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112448 2021-01-15T19:11:37.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207518+0000", 2021-01-15T19:11:37.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031650218000000001 2021-01-15T19:11:37.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.239168+0000", 2021-01-15T19:11:37.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035519202999999999 2021-01-15T19:11:37.077 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.077 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.077 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.078 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1292 3.8s0 3:125ea3c2:::smithi00140323-13:64 [read 0~6692864,checksum] snapc 0=[] ondisk+read+known_if_redirected e135)", 2021-01-15T19:11:37.079 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:37.079 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.575584133999996, 2021-01-15T19:11:37.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997, 2021-01-15T19:11:37.079 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.080 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.080 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.080 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1292 2021-01-15T19:11:37.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.081 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:37.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203760+0000", 2021-01-15T19:11:37.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010773900000000001 2021-01-15T19:11:37.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203868+0000", 2021-01-15T19:11:37.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5460000000000002e-06 2021-01-15T19:11:37.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203873+0000", 2021-01-15T19:11:37.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2110000000000001e-06 2021-01-15T19:11:37.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203880+0000", 2021-01-15T19:11:37.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6120000000000001e-06 2021-01-15T19:11:37.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203884+0000", 2021-01-15T19:11:37.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6671000000000002e-05 2021-01-15T19:11:37.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203921+0000", 2021-01-15T19:11:37.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0299999999999999e-06 2021-01-15T19:11:37.090 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.091 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:37.091 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.203927+0000", 2021-01-15T19:11:37.091 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038037370000000002 2021-01-15T19:11:37.091 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.092 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.092 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.092 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207731+0000", 2021-01-15T19:11:37.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013777099999999999 2021-01-15T19:11:37.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.093 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.094 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.207869+0000", 2021-01-15T19:11:37.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035974610999999997 2021-01-15T19:11:37.094 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.243843+0000", 2021-01-15T19:11:37.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040082927999999997 2021-01-15T19:11:37.095 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.096 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.098 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1581 3.8s0 3:125ea3c2:::smithi00140323-13:head [rollback 4b] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:37.098 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:37.098 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.483809812000004, 2021-01-15T19:11:37.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001, 2021-01-15T19:11:37.098 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.099 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.099 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.099 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.099 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.099 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1581 2021-01-15T19:11:37.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.100 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:37.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.295535+0000", 2021-01-15T19:11:37.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011929600000000001 2021-01-15T19:11:37.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296728+0000", 2021-01-15T19:11:37.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7250000000000003e-06 2021-01-15T19:11:37.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296736+0000", 2021-01-15T19:11:37.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8042000000000001e-05 2021-01-15T19:11:37.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296754+0000", 2021-01-15T19:11:37.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6090000000000001e-06 2021-01-15T19:11:37.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296763+0000", 2021-01-15T19:11:37.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096941330000000006 2021-01-15T19:11:37.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306457+0000", 2021-01-15T19:11:37.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014483200000000001 2021-01-15T19:11:37.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.306602+0000", 2021-01-15T19:11:37.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00077699300000000002 2021-01-15T19:11:37.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.307379+0000", 2021-01-15T19:11:37.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014122791000000001 2021-01-15T19:11:37.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321502+0000", 2021-01-15T19:11:37.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7976e-05 2021-01-15T19:11:37.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321580+0000", 2021-01-15T19:11:37.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114434 2021-01-15T19:11:37.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.321694+0000", 2021-01-15T19:11:37.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026159495000000001 2021-01-15T19:11:37.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.113 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.114 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1582 3.8s0 3:125ea3c2:::smithi00140323-13:head [append 0~0] snapc 82=[82,7d,7b,7a,78,72,70,6f,6b,67,66,64,63,56,4b] ondisk+write+known_if_redirected e159)", 2021-01-15T19:11:37.114 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:37.114 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.482526672999995, 2021-01-15T19:11:37.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998, 2021-01-15T19:11:37.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.115 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.116 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1582 2021-01-15T19:11:37.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.116 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:37.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296818+0000", 2021-01-15T19:11:37.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0534000000000002e-05 2021-01-15T19:11:37.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296858+0000", 2021-01-15T19:11:37.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2455e-05 2021-01-15T19:11:37.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296871+0000", 2021-01-15T19:11:37.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3361e-05 2021-01-15T19:11:37.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296884+0000", 2021-01-15T19:11:37.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1670000000000004e-06 2021-01-15T19:11:37.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.296890+0000", 2021-01-15T19:11:37.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018158174999999999 2021-01-15T19:11:37.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315048+0000", 2021-01-15T19:11:37.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015273399999999999 2021-01-15T19:11:37.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315201+0000", 2021-01-15T19:11:37.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059205900000000003 2021-01-15T19:11:37.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.315793+0000", 2021-01-15T19:11:37.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006522562 2021-01-15T19:11:37.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322316+0000", 2021-01-15T19:11:37.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9053000000000004e-05 2021-01-15T19:11:37.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.322395+0000", 2021-01-15T19:11:37.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058830660000000002 2021-01-15T19:11:37.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:27.328278+0000", 2021-01-15T19:11:37.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031460165999999998 2021-01-15T19:11:37.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.129 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.130 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1936 3.8s0 3:125ea3c2:::smithi00140323-13:head [read 0~8306688,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:37.130 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:37.130 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.577398705, 2021-01-15T19:11:37.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001, 2021-01-15T19:11:37.131 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.131 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.131 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.131 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.132 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.132 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1936 2021-01-15T19:11:37.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.132 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:37.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.201946+0000", 2021-01-15T19:11:37.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4086999999999999e-05 2021-01-15T19:11:37.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202000+0000", 2021-01-15T19:11:37.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0649999999999999e-06 2021-01-15T19:11:37.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202008+0000", 2021-01-15T19:11:37.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2016e-05 2021-01-15T19:11:37.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202020+0000", 2021-01-15T19:11:37.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7890000000000002e-06 2021-01-15T19:11:37.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.202025+0000", 2021-01-15T19:11:37.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073266779999999997 2021-01-15T19:11:37.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209351+0000", 2021-01-15T19:11:37.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000145029 2021-01-15T19:11:37.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.209496+0000", 2021-01-15T19:11:37.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017658297999999999 2021-01-15T19:11:37.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:50.227155+0000", 2021-01-15T19:11:37.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025208962000000001 2021-01-15T19:11:37.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.142 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:37.144 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:37.144 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:37.143+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:37.245 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:37.243+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:37.258 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is unset 2021-01-15T19:11:37.320 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:37.321 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:37.322 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:37.329 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:37.345 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:37.344+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:37.446 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:37.445+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:37.493 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:37.494 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:37.495 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:37.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:37.547 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:37.546+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:37.649 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:37.647+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:37.671 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:37.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:11:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 193.38326968300001, 2021-01-15T19:11:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:11:37.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:11:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:11:37.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:11:37.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:11:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:11:37.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:11:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:11:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:11:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:11:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:11:37.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:11:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:11:37.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:11:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:11:37.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:11:37.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:11:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:11:37.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:11:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:11:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:11:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:11:37.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:11:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:11:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:11:37.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:11:37.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:11:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.45150691699999, 2021-01-15T19:11:37.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:11:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:11:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.696 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:11:37.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:11:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:11:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:11:37.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:11:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:11:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:11:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:11:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:11:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:11:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:11:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:11:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:11:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:11:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:11:37.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:11:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:11:37.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:11:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:11:37.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:11:37.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:11:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:11:37.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:11:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:11:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:11:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:37.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.449218876, 2021-01-15T19:11:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:11:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:11:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:37.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:11:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:11:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:11:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:11:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:11:37.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:11:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:11:37.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:11:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:11:37.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:11:37.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:11:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:11:37.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:11:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:11:37.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:11:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:11:37.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:11:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:11:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:11:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:11:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:11:37.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.44662925099999, 2021-01-15T19:11:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:11:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.736 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:11:37.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:11:37.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:11:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:11:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:11:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:11:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:11:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:11:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:11:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:11:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:11:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:11:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:11:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:11:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:11:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:11:37.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:11:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:11:37.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:11:37.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:11:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:11:37.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:11:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:11:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.44572097299999, 2021-01-15T19:11:37.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:11:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:11:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:11:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:11:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:11:37.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:11:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:11:37.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:11:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:11:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:11:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:11:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:11:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:11:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:11:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:11:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:11:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:11:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:11:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:11:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:11:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:11:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.769 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:37.770 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 184.47703706600001, 2021-01-15T19:11:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:11:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:11:37.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:37.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:37.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:11:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:11:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:11:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:11:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:11:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:11:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:11:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:11:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:11:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:11:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:11:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:11:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:11:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:11:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:11:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:11:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:11:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:11:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:11:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:11:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:11:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:11:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:11:37.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:11:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:11:37.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 184.47688732899999, 2021-01-15T19:11:37.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:11:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:11:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:37.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:11:37.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:11:37.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:11:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:11:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:11:37.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:11:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:11:37.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:11:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:11:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:37.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:11:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:11:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:11:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:11:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:11:37.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:11:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:11:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:11:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:11:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:11:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:11:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:11:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:11:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:11:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:11:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:11:37.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:37.809 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 184.47673543100001, 2021-01-15T19:11:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:11:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.811 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:11:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.812 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:37.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:11:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:11:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:11:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:11:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:11:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:11:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:11:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:11:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:11:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:11:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:11:37.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:11:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:11:37.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:11:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:11:37.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:11:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:11:37.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:11:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:11:37.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:11:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:11:37.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:11:37.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:11:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:11:37.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:11:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 184.47661011899999, 2021-01-15T19:11:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:11:37.829 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.830 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:11:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:11:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:11:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:11:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:11:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:11:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:11:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:11:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:11:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:11:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:11:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:11:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:11:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:11:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:11:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:11:37.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:11:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:11:37.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:11:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:11:37.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:11:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:11:37.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:11:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:11:37.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:11:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:11:37.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:11:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:11:37.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.847 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:11:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.455777544, 2021-01-15T19:11:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:11:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.848 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.849 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:11:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:11:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:11:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:11:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:11:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:11:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:11:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:11:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:11:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:11:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:11:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:11:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:11:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:11:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:11:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:11:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:11:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:11:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:11:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:11:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:11:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:11:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.862 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:11:37.863 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.44948421199999, 2021-01-15T19:11:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:11:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:11:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.865 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:37.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:37.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:11:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:11:37.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:11:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:11:37.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:11:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:11:37.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:11:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:11:37.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:11:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:11:37.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:11:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:11:37.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:11:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:11:37.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:11:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:11:37.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:11:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:11:37.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:11:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:11:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 178.487424699, 2021-01-15T19:11:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:11:37.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.879 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:11:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:11:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:11:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:11:37.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:11:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:11:37.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:11:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:11:37.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:11:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:11:37.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:11:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:11:37.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:11:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:11:37.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:11:37.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:11:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:11:37.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:11:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:11:37.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:11:37.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:11:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:11:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:11:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:11:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:11:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:11:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:37.898 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 178.42505271900001, 2021-01-15T19:11:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:11:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.899 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:11:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.900 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:11:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:11:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:11:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:11:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:11:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:11:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:11:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:11:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:11:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:11:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:11:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:11:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:11:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:11:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:11:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:11:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:11:37.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:11:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:11:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:11:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:11:37.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 178.421750218, 2021-01-15T19:11:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:11:37.916 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.917 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:11:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:11:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:11:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:11:37.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:11:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:11:37.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:11:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:11:37.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:11:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:11:37.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:11:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:11:37.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:11:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:11:37.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:11:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:37.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:11:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:11:37.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:11:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:11:37.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:11:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:37.932 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 178.42042835199999, 2021-01-15T19:11:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:11:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.933 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:11:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:11:37.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:11:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:11:37.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:11:37.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:11:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:11:37.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:11:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:11:37.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:11:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:11:37.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:11:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:11:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:11:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:11:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:11:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:11:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:11:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:11:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:11:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:11:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:37.948 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 178.42005809899999, 2021-01-15T19:11:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:11:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.949 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:11:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.950 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:11:37.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:11:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:11:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:11:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:11:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:11:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:11:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:11:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:11:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:11:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:11:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:11:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:11:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:11:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:11:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:11:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:11:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:11:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:11:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:11:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:11:37.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.964 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 178.41972370600001, 2021-01-15T19:11:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:11:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.965 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.966 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:11:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:11:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:11:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:11:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:11:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:11:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:11:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:11:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:11:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:11:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:11:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:11:37.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:11:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:11:37.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:11:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:11:37.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:11:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:11:37.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:11:37.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:11:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:11:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 164.478462447, 2021-01-15T19:11:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:11:37.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:11:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:11:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:11:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:11:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:11:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:11:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:11:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:11:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:11:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:11:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:11:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:11:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:11:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:11:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:11:37.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:11:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:11:37.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:11:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:11:37.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:11:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:37.994 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 154.486099575, 2021-01-15T19:11:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:11:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:37.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:11:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.996 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:37.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:37.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:11:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:11:37.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:11:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:11:38.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:11:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:11:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:11:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:11:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:11:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:11:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:11:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:38.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:11:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:11:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:11:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:11:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:11:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:11:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:11:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:11:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:11:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:11:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:11:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:11:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:11:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:11:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:11:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:38.013 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 154.45610537100001, 2021-01-15T19:11:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:11:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:11:38.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:11:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:11:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:11:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:11:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:11:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:11:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:11:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:11:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:11:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:11:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:11:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:11:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:11:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:11:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:11:38.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:11:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:11:38.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:11:38.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:11:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.029 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:38.030 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:38.031 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:38.029+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:38.131 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:38.130+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:38.197 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:38.197 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:38.198 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:38.207 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:38.248 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:38.247+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:38.333 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:38.332+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:38.373 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:38.373 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:38.374 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:38.374 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:38.382 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:38.434 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:38.433+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:38.536 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:38.534+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:38.549 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.550 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:11:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 179.362993894, 2021-01-15T19:11:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:11:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.551 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:11:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:11:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:11:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:11:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:11:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:11:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:11:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:11:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:11:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:11:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:11:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:11:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:11:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:11:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:11:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:11:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:11:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.562 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:38.563 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 175.31550571700001, 2021-01-15T19:11:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:11:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:38.564 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:11:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.565 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:11:38.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:11:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:11:38.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:11:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:11:38.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:11:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:11:38.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:11:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:11:38.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:11:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:11:38.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:11:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:11:38.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:11:38.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:11:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:11:38.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:11:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:38.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:11:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:11:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:11:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:11:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:11:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 175.315357699, 2021-01-15T19:11:38.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:11:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:11:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:38.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:11:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:11:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:11:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:11:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:11:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:11:38.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:11:38.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:11:38.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:11:38.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:11:38.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:11:38.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:38.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:11:38.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:11:38.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:11:38.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:11:38.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:38.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:11:38.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:11:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:11:38.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:11:38.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:11:38.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:11:38.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:11:38.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:11:38.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.599 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:38.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:38.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.361036708, 2021-01-15T19:11:38.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:11:38.602 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.602 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.603 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:38.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.603 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:38.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:38.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:11:38.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:11:38.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:11:38.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:11:38.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:11:38.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:11:38.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:11:38.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:11:38.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:11:38.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:11:38.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:11:38.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:38.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:11:38.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:11:38.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:11:38.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:11:38.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:11:38.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:11:38.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:11:38.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:11:38.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.616 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.617 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:38.617 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:38.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.36084541899999, 2021-01-15T19:11:38.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:11:38.618 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.618 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.619 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:38.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.620 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:38.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:38.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:11:38.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:11:38.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:11:38.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:11:38.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:11:38.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:11:38.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:11:38.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:11:38.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:11:38.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:11:38.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:11:38.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:38.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:11:38.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:11:38.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:11:38.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:11:38.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:11:38.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:11:38.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:11:38.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:11:38.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.633 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.634 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:38.634 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:38.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 140.36068306600001, 2021-01-15T19:11:38.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:11:38.635 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.635 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.636 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:38.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.636 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:38.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:38.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:11:38.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:11:38.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:11:38.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:11:38.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:11:38.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:11:38.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:11:38.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:11:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:11:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:11:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:11:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:11:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:11:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:11:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:11:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:11:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:11:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:11:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:11:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:38.649 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.32896315299999, 2021-01-15T19:11:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:11:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.650 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:11:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:11:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:11:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:11:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:11:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:11:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:11:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:11:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:11:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:11:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:11:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:11:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:11:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:11:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:11:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:11:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.661 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 134.32890146400001, 2021-01-15T19:11:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:11:38.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:11:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:38.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:11:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:11:38.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:11:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:11:38.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:11:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:11:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:11:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:11:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:11:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:11:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:11:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:11:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:11:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:11:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:11:38.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.676 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:11:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 97.224835126000002, 2021-01-15T19:11:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:11:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.677 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:11:38.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:11:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:11:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:11:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:11:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:11:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:11:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:11:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:11:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:11:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:11:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:11:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:11:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:11:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:11:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:11:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:11:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:11:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:11:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:11:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:11:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:11:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:11:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:11:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:11:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:11:38.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 97.224748554000001, 2021-01-15T19:11:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:11:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:11:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:11:38.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:11:38.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:11:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:11:38.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:11:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:11:38.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:11:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:11:38.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:11:38.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:11:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:38.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:11:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:11:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:11:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:11:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:11:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:11:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:11:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:11:38.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:11:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:11:38.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:11:38.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:11:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:11:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:11:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:11:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:11:38.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.720 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:11:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 95.334438425000002, 2021-01-15T19:11:38.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:11:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.722 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:11:38.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:11:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:11:38.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:11:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:11:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:11:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:11:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:11:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:11:38.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:11:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:11:38.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:11:38.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:11:38.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:11:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:11:38.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:11:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:11:38.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:11:38.745 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:38.746 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.209277576000005, 2021-01-15T19:11:38.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:11:38.747 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.747 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:38.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:11:38.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:38.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:38.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:11:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:11:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:11:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:11:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:38.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:11:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:11:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:11:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:11:38.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:38.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:11:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:11:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:11:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:11:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:11:38.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:11:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:11:38.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:11:38.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:11:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:11:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:11:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:11:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:38.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 92.209181655999998, 2021-01-15T19:11:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:11:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.772 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:11:38.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:11:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:11:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:11:38.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:11:38.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:11:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:11:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:11:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:11:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:11:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:11:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:11:38.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:11:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:11:38.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:11:38.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:11:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:38.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:11:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:11:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:11:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:11:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:11:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:11:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:11:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:11:38.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.790 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:11:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.342123315999999, 2021-01-15T19:11:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:11:38.791 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.792 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:11:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:11:38.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:11:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:11:38.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:11:38.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:11:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:11:38.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:11:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:11:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:11:38.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:11:38.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:11:38.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:11:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:11:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:11:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:11:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:11:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:11:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:11:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:11:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:11:38.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:11:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:11:38.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:11:38.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:11:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:11:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:38.811 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 87.342032239000005, 2021-01-15T19:11:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:11:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.812 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:11:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.813 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:38.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:38.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:11:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:11:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:11:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:11:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:11:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:11:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:11:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:11:38.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:11:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:11:38.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:11:38.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:11:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:11:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:11:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:38.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:11:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:11:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:11:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:11:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:11:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:11:38.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:11:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:11:38.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:11:38.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:11:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:11:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:11:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.831 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:11:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:38.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.332552984000003, 2021-01-15T19:11:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:11:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:38.833 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:11:38.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:38.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:11:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:11:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:11:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:11:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:11:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:11:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:11:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:11:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:11:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:11:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:11:38.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:11:38.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:11:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:11:38.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:11:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:11:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:11:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:11:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:11:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:38.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:11:38.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:11:38.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:11:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:11:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:11:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:11:38.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.851 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:11:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.332407959999998, 2021-01-15T19:11:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:11:38.852 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.853 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:11:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:11:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:11:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:11:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:11:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:11:38.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:11:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:11:38.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:11:38.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:11:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:11:38.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:11:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:38.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:11:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:11:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:11:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:11:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:11:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:11:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:11:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:11:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:11:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:11:38.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:11:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:11:38.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:11:38.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:11:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:11:38.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:11:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.166914087000002, 2021-01-15T19:11:38.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:11:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.873 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:11:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.874 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:38.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:11:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:11:38.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:11:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:11:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:11:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:11:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:11:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:11:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:11:38.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:11:38.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:11:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:11:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:11:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:11:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:11:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:38.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.166826659999998, 2021-01-15T19:11:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:11:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.887 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:11:38.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:38.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:11:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:11:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:11:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:11:38.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:11:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:11:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:11:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:11:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:11:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:11:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:11:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:11:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:11:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:11:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:11:38.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.900 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.166746695999997, 2021-01-15T19:11:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:11:38.901 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:38.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:11:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:11:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:11:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:11:38.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:11:38.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:11:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:11:38.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:11:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:11:38.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:11:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:38.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:11:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:11:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:38.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:11:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:11:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:38.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:11:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:11:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:38.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:38.914 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:38.915 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:38.917 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:38.917 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:38.916+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:39.016 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:39.015+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:39.077 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:39.083 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:39.083 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:39.083 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:39.093 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:39.118 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:39.116+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:39.119 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:39.139 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:39.160 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:39.181 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:39.181 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:39.182 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:39.182 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:39.182 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:39.182 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:39.183 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:39.183 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:39.183 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:39.183 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:39.184 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:39.184 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:39.185 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:39.225 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:39.224+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:39.265 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:39.266 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:39.274 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2021-01-15T19:11:39.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:39.321 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:39.320+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:39.403 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:39.403 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:39.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:39.454 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:39.453+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:39.520 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:39.518+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:39.545 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:39.545 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:39.555 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:39.621 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:39.620+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:39.722 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:39.721+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:39.725 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:39.725 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:39.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:39.823 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:39.822+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:39.908 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:39.908 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:39.918 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:39.923 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:39.923+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:40.024 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:40.023+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:40.089 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:40.090 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:40.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:40.141 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:40.140+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:40.221 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is unset 2021-01-15T19:11:40.225 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:40.224+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:40.258 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7, 5] out_osds: [6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:40.259 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:40.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2021-01-15T19:11:40.279 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:40.279 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:40.289 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:40.327 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:40.326+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:40.428 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:40.427+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:40.465 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:40.466 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:40.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:40.529 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:40.527+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:40.600 INFO:teuthology.orchestra.run.smithi001.stdout: 2021-01-15T19:11:40.600 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 212, 2021-01-15T19:11:40.601 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "fe0e1890-043f-40bf-ad11-c5f08349f418", 2021-01-15T19:11:40.602 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2021-01-15T19:07:43.378003+0000", 2021-01-15T19:11:40.602 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2021-01-15T19:11:40.212476+0000", 2021-01-15T19:11:40.602 INFO:teuthology.orchestra.run.smithi001.stdout: "last_up_change": "2021-01-15T19:10:45.255653+0000", 2021-01-15T19:11:40.603 INFO:teuthology.orchestra.run.smithi001.stdout: "last_in_change": "2021-01-15T19:11:29.213648+0000", 2021-01-15T19:11:40.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-01-15T19:11:40.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_num": 5799936, 2021-01-15T19:11:40.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_set": [ 2021-01-15T19:11:40.605 INFO:teuthology.orchestra.run.smithi001.stdout: "pglog_hardlimit", 2021-01-15T19:11:40.605 INFO:teuthology.orchestra.run.smithi001.stdout: "purged_snapdirs", 2021-01-15T19:11:40.606 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_deletes", 2021-01-15T19:11:40.606 INFO:teuthology.orchestra.run.smithi001.stdout: "sortbitwise" 2021-01-15T19:11:40.606 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:11:40.607 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_version": 5, 2021-01-15T19:11:40.607 INFO:teuthology.orchestra.run.smithi001.stdout: "full_ratio": 0.94999998807907104, 2021-01-15T19:11:40.608 INFO:teuthology.orchestra.run.smithi001.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-01-15T19:11:40.608 INFO:teuthology.orchestra.run.smithi001.stdout: "nearfull_ratio": 0.85000002384185791, 2021-01-15T19:11:40.608 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_snapshot": "", 2021-01-15T19:11:40.609 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_max": 3, 2021-01-15T19:11:40.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd": 8, 2021-01-15T19:11:40.610 INFO:teuthology.orchestra.run.smithi001.stdout: "require_min_compat_client": "luminous", 2021-01-15T19:11:40.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min_compat_client": "jewel", 2021-01-15T19:11:40.611 INFO:teuthology.orchestra.run.smithi001.stdout: "require_osd_release": "pacific", 2021-01-15T19:11:40.611 INFO:teuthology.orchestra.run.smithi001.stdout: "pools": [ 2021-01-15T19:11:40.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.612 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 1, 2021-01-15T19:11:40.613 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "device_health_metrics", 2021-01-15T19:11:40.613 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:47.545867+0000", 2021-01-15T19:11:40.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:11:40.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:11:40.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:11:40.615 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:11:40.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:11:40.616 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:11:40.616 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:11:40.616 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:11:40.616 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:11:40.617 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:11:40.617 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:11:40.617 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:11:40.617 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 21, 2021-01-15T19:11:40.617 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 21, 2021-01-15T19:11:40.618 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 21, 2021-01-15T19:11:40.618 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 21, 2021-01-15T19:11:40.618 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 21, 2021-01-15T19:11:40.618 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:11:40.619 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:11:40.619 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:11:40.619 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:11:40.619 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:11:40.620 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:11:40.620 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:11:40.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.621 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "196", 2021-01-15T19:11:40.621 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:11:40.621 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:11:40.621 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "154", 2021-01-15T19:11:40.622 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:11:40.622 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:11:40.622 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:11:40.622 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:11:40.623 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:11:40.623 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:11:40.623 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:11:40.623 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:11:40.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:11:40.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:11:40.624 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:11:40.624 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:11:40.624 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:11:40.624 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:11:40.625 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:11:40.625 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:11:40.625 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:11:40.625 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:11:40.626 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:11:40.626 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:11:40.626 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:11:40.626 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:11:40.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:11:40.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.627 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:11:40.627 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:11:40.628 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:11:40.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:11:40.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:11:40.628 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:11:40.628 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:11:40.629 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:11:40.629 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:11:40.629 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:11:40.629 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:11:40.630 INFO:teuthology.orchestra.run.smithi001.stdout: "options": { 2021-01-15T19:11:40.630 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_min": 1 2021-01-15T19:11:40.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.630 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:11:40.631 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_devicehealth": {} 2021-01-15T19:11:40.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.632 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 2, 2021-01-15T19:11:40.632 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "rbd", 2021-01-15T19:11:40.632 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:07:54.484638+0000", 2021-01-15T19:11:40.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2021-01-15T19:11:40.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool", 2021-01-15T19:11:40.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 1, 2021-01-15T19:11:40.633 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 2, 2021-01-15T19:11:40.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 1, 2021-01-15T19:11:40.634 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 0, 2021-01-15T19:11:40.634 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:11:40.634 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:11:40.634 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:11:40.634 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:11:40.635 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:11:40.635 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:11:40.635 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 28, 2021-01-15T19:11:40.635 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 28, 2021-01-15T19:11:40.636 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 28, 2021-01-15T19:11:40.636 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 28, 2021-01-15T19:11:40.636 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 28, 2021-01-15T19:11:40.636 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:11:40.636 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:11:40.637 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:11:40.637 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:11:40.637 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:11:40.637 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:11:40.638 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:11:40.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.638 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "210", 2021-01-15T19:11:40.638 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:11:40.639 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:11:40.639 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "208", 2021-01-15T19:11:40.639 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:11:40.639 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:11:40.640 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 0, 2021-01-15T19:11:40.640 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 0, 2021-01-15T19:11:40.640 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:11:40.640 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:11:40.641 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:11:40.641 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:11:40.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:11:40.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:11:40.642 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:11:40.642 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:11:40.642 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:11:40.642 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:11:40.643 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:11:40.643 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:11:40.643 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:11:40.643 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:11:40.644 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:11:40.644 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:11:40.644 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "", 2021-01-15T19:11:40.644 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:11:40.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:11:40.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.645 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:11:40.645 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:11:40.645 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:11:40.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:11:40.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:11:40.646 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:11:40.646 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:11:40.647 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:11:40.647 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 0, 2021-01-15T19:11:40.647 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:11:40.647 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:11:40.647 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:11:40.648 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:11:40.648 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd": {} 2021-01-15T19:11:40.648 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.649 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:11:40.649 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": "unique_pool_0", 2021-01-15T19:11:40.649 INFO:teuthology.orchestra.run.smithi001.stdout: "create_time": "2021-01-15T19:08:08.242953+0000", 2021-01-15T19:11:40.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 8193, 2021-01-15T19:11:40.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2021-01-15T19:11:40.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": 3, 2021-01-15T19:11:40.650 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 3, 2021-01-15T19:11:40.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": 2, 2021-01-15T19:11:40.651 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_rule": 1, 2021-01-15T19:11:40.651 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_count": 0, 2021-01-15T19:11:40.651 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_target": 0, 2021-01-15T19:11:40.652 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_barrier": 0, 2021-01-15T19:11:40.652 INFO:teuthology.orchestra.run.smithi001.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2021-01-15T19:11:40.652 INFO:teuthology.orchestra.run.smithi001.stdout: "object_hash": 2, 2021-01-15T19:11:40.652 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscale_mode": "off", 2021-01-15T19:11:40.652 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": 26, 2021-01-15T19:11:40.653 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num": 26, 2021-01-15T19:11:40.653 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_placement_num_target": 26, 2021-01-15T19:11:40.653 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_target": 26, 2021-01-15T19:11:40.653 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num_pending": 26, 2021-01-15T19:11:40.654 INFO:teuthology.orchestra.run.smithi001.stdout: "last_pg_merge_meta": { 2021-01-15T19:11:40.654 INFO:teuthology.orchestra.run.smithi001.stdout: "source_pgid": "0.0", 2021-01-15T19:11:40.654 INFO:teuthology.orchestra.run.smithi001.stdout: "ready_epoch": 0, 2021-01-15T19:11:40.654 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_started": 0, 2021-01-15T19:11:40.655 INFO:teuthology.orchestra.run.smithi001.stdout: "last_epoch_clean": 0, 2021-01-15T19:11:40.655 INFO:teuthology.orchestra.run.smithi001.stdout: "source_version": "0'0", 2021-01-15T19:11:40.655 INFO:teuthology.orchestra.run.smithi001.stdout: "target_version": "0'0" 2021-01-15T19:11:40.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.655 INFO:teuthology.orchestra.run.smithi001.stdout: "last_change": "181", 2021-01-15T19:11:40.656 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend": "0", 2021-01-15T19:11:40.656 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_prenautilus": "0", 2021-01-15T19:11:40.656 INFO:teuthology.orchestra.run.smithi001.stdout: "last_force_op_resend_preluminous": "52", 2021-01-15T19:11:40.656 INFO:teuthology.orchestra.run.smithi001.stdout: "auid": 0, 2021-01-15T19:11:40.657 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_mode": "selfmanaged", 2021-01-15T19:11:40.657 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_seq": 152, 2021-01-15T19:11:40.657 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_epoch": 181, 2021-01-15T19:11:40.657 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_snaps": [], 2021-01-15T19:11:40.657 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps": "[]", 2021-01-15T19:11:40.658 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_bytes": 0, 2021-01-15T19:11:40.658 INFO:teuthology.orchestra.run.smithi001.stdout: "quota_max_objects": 0, 2021-01-15T19:11:40.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tiers": [], 2021-01-15T19:11:40.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tier_of": -1, 2021-01-15T19:11:40.659 INFO:teuthology.orchestra.run.smithi001.stdout: "read_tier": -1, 2021-01-15T19:11:40.659 INFO:teuthology.orchestra.run.smithi001.stdout: "write_tier": -1, 2021-01-15T19:11:40.659 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_mode": "none", 2021-01-15T19:11:40.659 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_bytes": 0, 2021-01-15T19:11:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "target_max_objects": 0, 2021-01-15T19:11:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-01-15T19:11:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-01-15T19:11:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_target_full_ratio_micro": 800000, 2021-01-15T19:11:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_flush_age": 0, 2021-01-15T19:11:40.661 INFO:teuthology.orchestra.run.smithi001.stdout: "cache_min_evict_age": 0, 2021-01-15T19:11:40.661 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profile": "isaprofile", 2021-01-15T19:11:40.661 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_params": { 2021-01-15T19:11:40.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "none" 2021-01-15T19:11:40.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.662 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_period": 0, 2021-01-15T19:11:40.662 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_count": 0, 2021-01-15T19:11:40.662 INFO:teuthology.orchestra.run.smithi001.stdout: "use_gmt_hitset": true, 2021-01-15T19:11:40.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min_read_recency_for_promote": 0, 2021-01-15T19:11:40.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min_write_recency_for_promote": 0, 2021-01-15T19:11:40.663 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_grade_decay_rate": 0, 2021-01-15T19:11:40.663 INFO:teuthology.orchestra.run.smithi001.stdout: "hit_set_search_last_n": 0, 2021-01-15T19:11:40.664 INFO:teuthology.orchestra.run.smithi001.stdout: "grade_table": [], 2021-01-15T19:11:40.664 INFO:teuthology.orchestra.run.smithi001.stdout: "stripe_width": 8192, 2021-01-15T19:11:40.664 INFO:teuthology.orchestra.run.smithi001.stdout: "expected_num_objects": 0, 2021-01-15T19:11:40.664 INFO:teuthology.orchestra.run.smithi001.stdout: "fast_read": false, 2021-01-15T19:11:40.664 INFO:teuthology.orchestra.run.smithi001.stdout: "options": {}, 2021-01-15T19:11:40.665 INFO:teuthology.orchestra.run.smithi001.stdout: "application_metadata": { 2021-01-15T19:11:40.665 INFO:teuthology.orchestra.run.smithi001.stdout: "rados": {} 2021-01-15T19:11:40.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.666 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:11:40.666 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.666 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:11:40.666 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "e5b26f89-5c99-4764-9183-f3bb27c118f5", 2021-01-15T19:11:40.667 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 0, 2021-01-15T19:11:40.667 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.667 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:11:40.667 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.668 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.668 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.668 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.668 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 159, 2021-01-15T19:11:40.669 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 177, 2021-01-15T19:11:40.669 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.669 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.669 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.670 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2021-01-15T19:11:40.670 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.671 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2021-01-15T19:11:40.671 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.672 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.672 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.672 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.673 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6802", 2021-01-15T19:11:40.673 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.674 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6803", 2021-01-15T19:11:40.675 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.675 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.675 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.676 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.676 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.676 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6806", 2021-01-15T19:11:40.677 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.678 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6807", 2021-01-15T19:11:40.678 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.678 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.678 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.679 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.679 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.680 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6804", 2021-01-15T19:11:40.680 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.681 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6805", 2021-01-15T19:11:40.681 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34833 2021-01-15T19:11:40.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.682 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6801/34833", 2021-01-15T19:11:40.682 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6803/34833", 2021-01-15T19:11:40.683 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6807/34833", 2021-01-15T19:11:40.683 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6805/34833", 2021-01-15T19:11:40.683 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.683 INFO:teuthology.orchestra.run.smithi001.stdout: "exists" 2021-01-15T19:11:40.684 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.684 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:11:40.684 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "99c0d455-1bd6-43aa-bfec-b6bddf87a35e", 2021-01-15T19:11:40.685 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 0, 2021-01-15T19:11:40.685 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.685 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:11:40.685 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 0.052215576171875, 2021-01-15T19:11:40.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.686 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.686 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.686 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 158, 2021-01-15T19:11:40.687 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 159, 2021-01-15T19:11:40.687 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.687 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.687 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.687 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.688 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6808", 2021-01-15T19:11:40.688 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.689 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6809", 2021-01-15T19:11:40.689 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.690 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.690 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.690 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.690 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.691 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6810", 2021-01-15T19:11:40.691 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.692 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6811", 2021-01-15T19:11:40.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.694 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.695 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6814", 2021-01-15T19:11:40.695 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.696 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6815", 2021-01-15T19:11:40.696 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.696 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.697 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.697 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.698 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6812", 2021-01-15T19:11:40.698 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6813", 2021-01-15T19:11:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34834 2021-01-15T19:11:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6809/34834", 2021-01-15T19:11:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6811/34834", 2021-01-15T19:11:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6815/34834", 2021-01-15T19:11:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6813/34834", 2021-01-15T19:11:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "exists" 2021-01-15T19:11:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:11:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "71fe6c4b-3d73-4e07-b49d-98596ca7d9da", 2021-01-15T19:11:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:11:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:11:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 210, 2021-01-15T19:11:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:11:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6817", 2021-01-15T19:11:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6819", 2021-01-15T19:11:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6821", 2021-01-15T19:11:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6823", 2021-01-15T19:11:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.713 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6830", 2021-01-15T19:11:40.713 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.714 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6831", 2021-01-15T19:11:40.715 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.715 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.715 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.716 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.717 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6826", 2021-01-15T19:11:40.717 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.718 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6828", 2021-01-15T19:11:40.718 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34863 2021-01-15T19:11:40.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.718 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.719 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6819/34863", 2021-01-15T19:11:40.719 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6823/34863", 2021-01-15T19:11:40.719 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6831/34863", 2021-01-15T19:11:40.720 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6828/34863", 2021-01-15T19:11:40.720 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.720 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:11:40.720 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:11:40.721 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.721 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:11:40.721 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "8a6b018a-3a48-4a01-8526-a386f62ecb36", 2021-01-15T19:11:40.722 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:11:40.722 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.722 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0.198638916015625, 2021-01-15T19:11:40.722 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.723 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.723 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.723 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.723 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 210, 2021-01-15T19:11:40.724 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:11:40.724 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.724 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.724 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.725 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6816", 2021-01-15T19:11:40.725 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.726 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6818", 2021-01-15T19:11:40.727 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.727 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.728 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.728 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.729 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6820", 2021-01-15T19:11:40.729 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.730 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6822", 2021-01-15T19:11:40.730 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.730 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.730 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.731 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.731 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.731 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.732 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6827", 2021-01-15T19:11:40.732 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.733 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6829", 2021-01-15T19:11:40.733 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.734 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6824", 2021-01-15T19:11:40.735 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.736 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6825", 2021-01-15T19:11:40.737 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34864 2021-01-15T19:11:40.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.737 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.737 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6818/34864", 2021-01-15T19:11:40.738 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.1:6822/34864", 2021-01-15T19:11:40.738 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.1:6829/34864", 2021-01-15T19:11:40.738 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.1:6825/34864", 2021-01-15T19:11:40.738 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.739 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:11:40.739 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:11:40.739 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.740 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:11:40.740 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "ff089622-e642-4bd0-901c-60efdb87a549", 2021-01-15T19:11:40.740 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:11:40.741 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.741 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:11:40.741 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.741 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.741 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.742 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.742 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 208, 2021-01-15T19:11:40.742 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:11:40.742 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.743 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.743 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.744 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6800", 2021-01-15T19:11:40.744 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.745 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6801", 2021-01-15T19:11:40.745 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.746 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.746 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.747 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6802", 2021-01-15T19:11:40.747 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.748 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6803", 2021-01-15T19:11:40.748 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.749 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.749 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.750 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.750 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6806", 2021-01-15T19:11:40.751 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.751 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6807", 2021-01-15T19:11:40.752 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.752 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.752 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.753 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6804", 2021-01-15T19:11:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6805", 2021-01-15T19:11:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34829 2021-01-15T19:11:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6801/34829", 2021-01-15T19:11:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6803/34829", 2021-01-15T19:11:40.756 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6807/34829", 2021-01-15T19:11:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6805/34829", 2021-01-15T19:11:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:11:40.757 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:11:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.758 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:11:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "3c57c228-dfaa-4e33-bded-27da0048d141", 2021-01-15T19:11:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:11:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:11:40.759 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.760 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 211, 2021-01-15T19:11:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:11:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.761 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6809", 2021-01-15T19:11:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6811", 2021-01-15T19:11:40.763 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.764 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6813", 2021-01-15T19:11:40.765 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.766 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6815", 2021-01-15T19:11:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.767 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6821", 2021-01-15T19:11:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6823", 2021-01-15T19:11:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.770 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6817", 2021-01-15T19:11:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6819", 2021-01-15T19:11:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34830 2021-01-15T19:11:40.773 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6811/34830", 2021-01-15T19:11:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6815/34830", 2021-01-15T19:11:40.774 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6823/34830", 2021-01-15T19:11:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6819/34830", 2021-01-15T19:11:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:11:40.775 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:11:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.776 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:11:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "201f889f-dac1-4a26-a285-7a0ec21c3646", 2021-01-15T19:11:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:11:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 0, 2021-01-15T19:11:40.777 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2021-01-15T19:11:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 13, 2021-01-15T19:11:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 94, 2021-01-15T19:11:40.778 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 132, 2021-01-15T19:11:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 92, 2021-01-15T19:11:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 95, 2021-01-15T19:11:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.779 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6808", 2021-01-15T19:11:40.780 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6810", 2021-01-15T19:11:40.781 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.782 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.783 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6812", 2021-01-15T19:11:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6814", 2021-01-15T19:11:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6820", 2021-01-15T19:11:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6822", 2021-01-15T19:11:40.788 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.789 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6816", 2021-01-15T19:11:40.790 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6818", 2021-01-15T19:11:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 48096 2021-01-15T19:11:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.792 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6810/48096", 2021-01-15T19:11:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6814/48096", 2021-01-15T19:11:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6822/48096", 2021-01-15T19:11:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6818/48096", 2021-01-15T19:11:40.793 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:11:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:11:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:11:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "uuid": "79ffd413-a9d3-4180-9691-d834d89c3f2e", 2021-01-15T19:11:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "up": 1, 2021-01-15T19:11:40.795 INFO:teuthology.orchestra.run.smithi001.stdout: "in": 1, 2021-01-15T19:11:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 1, 2021-01-15T19:11:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_affinity": 1, 2021-01-15T19:11:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_begin": 0, 2021-01-15T19:11:40.796 INFO:teuthology.orchestra.run.smithi001.stdout: "last_clean_end": 0, 2021-01-15T19:11:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: "up_from": 13, 2021-01-15T19:11:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: "up_thru": 211, 2021-01-15T19:11:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: "down_at": 0, 2021-01-15T19:11:40.797 INFO:teuthology.orchestra.run.smithi001.stdout: "lost_at": 0, 2021-01-15T19:11:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2021-01-15T19:11:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6824", 2021-01-15T19:11:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6825", 2021-01-15T19:11:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addrs": { 2021-01-15T19:11:40.801 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6826", 2021-01-15T19:11:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6827", 2021-01-15T19:11:40.803 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.804 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addrs": { 2021-01-15T19:11:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6830", 2021-01-15T19:11:40.805 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.806 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6831", 2021-01-15T19:11:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.807 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addrs": { 2021-01-15T19:11:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2021-01-15T19:11:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2021-01-15T19:11:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6828", 2021-01-15T19:11:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2021-01-15T19:11:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.136:6829", 2021-01-15T19:11:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 34874 2021-01-15T19:11:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.810 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.136:6825/34874", 2021-01-15T19:11:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_addr": "172.21.15.136:6827/34874", 2021-01-15T19:11:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_back_addr": "172.21.15.136:6831/34874", 2021-01-15T19:11:40.811 INFO:teuthology.orchestra.run.smithi001.stdout: "heartbeat_front_addr": "172.21.15.136:6829/34874", 2021-01-15T19:11:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: "state": [ 2021-01-15T19:11:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: "exists", 2021-01-15T19:11:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: "up" 2021-01-15T19:11:40.812 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:11:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_xinfo": [ 2021-01-15T19:11:40.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 0, 2021-01-15T19:11:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.814 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:49.068217+0000", 2021-01-15T19:11:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 177 2021-01-15T19:11:40.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 1, 2021-01-15T19:11:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.816 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:09:49.114373+0000", 2021-01-15T19:11:40.817 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 159 2021-01-15T19:11:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.818 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 2, 2021-01-15T19:11:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.819 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:10:51.570589+0000", 2021-01-15T19:11:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:11:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 3, 2021-01-15T19:11:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.821 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:10:50.239826+0000", 2021-01-15T19:11:40.822 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:11:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 4, 2021-01-15T19:11:40.823 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.824 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:10:50.335135+0000", 2021-01-15T19:11:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:11:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.825 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 5, 2021-01-15T19:11:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.826 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:10:51.341578+0000", 2021-01-15T19:11:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:11:40.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 6, 2021-01-15T19:11:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.828 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 65536, 2021-01-15T19:11:40.829 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:11:01.438173+0000", 2021-01-15T19:11:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 95 2021-01-15T19:11:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.830 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": 7, 2021-01-15T19:11:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "down_stamp": "0.000000", 2021-01-15T19:11:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_probability": 0, 2021-01-15T19:11:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "laggy_interval": 0, 2021-01-15T19:11:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "features": 4540138297136906239, 2021-01-15T19:11:40.831 INFO:teuthology.orchestra.run.smithi001.stdout: "old_weight": 0, 2021-01-15T19:11:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: "last_purged_snaps_scrub": "2021-01-15T19:10:51.328325+0000", 2021-01-15T19:11:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: "dead_epoch": 0 2021-01-15T19:11:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.832 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:11:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap": [], 2021-01-15T19:11:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_upmap_items": [], 2021-01-15T19:11:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_temp": [ 2021-01-15T19:11:40.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "1.0", 2021-01-15T19:11:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2021-01-15T19:11:40.834 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "1.d", 2021-01-15T19:11:40.835 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: 2, 2021-01-15T19:11:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: 4 2021-01-15T19:11:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "1.14", 2021-01-15T19:11:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.837 INFO:teuthology.orchestra.run.smithi001.stdout: 7 2021-01-15T19:11:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.a", 2021-01-15T19:11:40.838 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2021-01-15T19:11:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.c", 2021-01-15T19:11:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.840 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: 7 2021-01-15T19:11:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.d", 2021-01-15T19:11:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.842 INFO:teuthology.orchestra.run.smithi001.stdout: 7 2021-01-15T19:11:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.843 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.13", 2021-01-15T19:11:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: 7, 2021-01-15T19:11:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2021-01-15T19:11:40.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.1a", 2021-01-15T19:11:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.845 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2021-01-15T19:11:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: "pgid": "2.1b", 2021-01-15T19:11:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: "osds": [ 2021-01-15T19:11:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: 5, 2021-01-15T19:11:40.847 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2021-01-15T19:11:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:11:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: "primary_temp": [], 2021-01-15T19:11:40.848 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklist": {}, 2021-01-15T19:11:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "erasure_code_profiles": { 2021-01-15T19:11:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default": { 2021-01-15T19:11:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:11:40.849 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:11:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:11:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "jerasure", 2021-01-15T19:11:40.850 INFO:teuthology.orchestra.run.smithi001.stdout: "ruleset-failure-domain": "osd", 2021-01-15T19:11:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:11:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "isaprofile": { 2021-01-15T19:11:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-device-class": "", 2021-01-15T19:11:40.851 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-failure-domain": "osd", 2021-01-15T19:11:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-root": "default", 2021-01-15T19:11:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: "k": "2", 2021-01-15T19:11:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: "m": "1", 2021-01-15T19:11:40.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "isaprofile", 2021-01-15T19:11:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: "plugin": "isa", 2021-01-15T19:11:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: "technique": "reed_sol_van" 2021-01-15T19:11:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_snaps_queue": [ 2021-01-15T19:11:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "pool": 3, 2021-01-15T19:11:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: "snaps": [ 2021-01-15T19:11:40.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 25, 2021-01-15T19:11:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 38, 2021-01-15T19:11:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.856 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 42, 2021-01-15T19:11:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.857 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 47, 2021-01-15T19:11:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 56, 2021-01-15T19:11:40.858 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 58, 2021-01-15T19:11:40.859 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 62, 2021-01-15T19:11:40.860 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 65, 2021-01-15T19:11:40.861 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 68, 2021-01-15T19:11:40.862 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 70, 2021-01-15T19:11:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 75, 2021-01-15T19:11:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 80, 2021-01-15T19:11:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 83, 2021-01-15T19:11:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 86, 2021-01-15T19:11:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 4 2021-01-15T19:11:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.867 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 92, 2021-01-15T19:11:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.868 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 95, 2021-01-15T19:11:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 98, 2021-01-15T19:11:40.869 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 102, 2021-01-15T19:11:40.870 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.871 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 109, 2021-01-15T19:11:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.872 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 112, 2021-01-15T19:11:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 4 2021-01-15T19:11:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 120, 2021-01-15T19:11:40.873 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 2 2021-01-15T19:11:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 125, 2021-01-15T19:11:40.874 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 132, 2021-01-15T19:11:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 141, 2021-01-15T19:11:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: "begin": 143, 2021-01-15T19:11:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: "length": 1 2021-01-15T19:11:40.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2021-01-15T19:11:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: "new_removed_snaps": [], 2021-01-15T19:11:40.878 INFO:teuthology.orchestra.run.smithi001.stdout: "new_purged_snaps": [], 2021-01-15T19:11:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_node_flags": {}, 2021-01-15T19:11:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: "device_class_flags": {}, 2021-01-15T19:11:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": { 2021-01-15T19:11:40.879 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_enabled": false, 2021-01-15T19:11:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_bucket_count": 0, 2021-01-15T19:11:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "degraded_stretch_mode": 0, 2021-01-15T19:11:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "recovering_stretch_mode": 0, 2021-01-15T19:11:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode_bucket": 0 2021-01-15T19:11:40.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:40.881 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:40.883 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:40.883 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:40.883 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:40.884 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:40.881+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:40.885 INFO:tasks.thrashosds.thrasher:j is {'epoch': 212, 'fsid': 'fe0e1890-043f-40bf-ad11-c5f08349f418', 'created': '2021-01-15T19:07:43.378003+0000', 'modified': '2021-01-15T19:11:40.212476+0000', 'last_up_change': '2021-01-15T19:10:45.255653+0000', 'last_in_change': '2021-01-15T19:11:29.213648+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-01-15T19:07:47.545867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '154', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-01-15T19:07:54.484638+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 28, 'pg_placement_num': 28, 'pg_placement_num_target': 28, 'pg_num_target': 28, 'pg_num_pending': 28, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '210', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '208', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2021-01-15T19:08:08.242953+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '52', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 152, 'snap_epoch': 181, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'isaprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'e5b26f89-5c99-4764-9183-f3bb27c118f5', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 159, 'down_at': 177, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6800', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6801', 'nonce': 34833}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6802', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6803', 'nonce': 34833}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6806', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6807', 'nonce': 34833}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6804', 'nonce': 34833}, {'type': 'v1', 'addr': '172.21.15.1:6805', 'nonce': 34833}]}, 'public_addr': '172.21.15.1:6801/34833', 'cluster_addr': '172.21.15.1:6803/34833', 'heartbeat_back_addr': '172.21.15.1:6807/34833', 'heartbeat_front_addr': '172.21.15.1:6805/34833', 'state': ['exists']}, {'osd': 1, 'uuid': '99c0d455-1bd6-43aa-bfec-b6bddf87a35e', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0.052215576171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 158, 'down_at': 159, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6808', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6809', 'nonce': 34834}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6810', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6811', 'nonce': 34834}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6814', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6815', 'nonce': 34834}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6812', 'nonce': 34834}, {'type': 'v1', 'addr': '172.21.15.1:6813', 'nonce': 34834}]}, 'public_addr': '172.21.15.1:6809/34834', 'cluster_addr': '172.21.15.1:6811/34834', 'heartbeat_back_addr': '172.21.15.1:6815/34834', 'heartbeat_front_addr': '172.21.15.1:6813/34834', 'state': ['exists']}, {'osd': 2, 'uuid': '71fe6c4b-3d73-4e07-b49d-98596ca7d9da', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 210, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6817', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6819', 'nonce': 34863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6821', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6823', 'nonce': 34863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6830', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6831', 'nonce': 34863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6826', 'nonce': 34863}, {'type': 'v1', 'addr': '172.21.15.1:6828', 'nonce': 34863}]}, 'public_addr': '172.21.15.1:6819/34863', 'cluster_addr': '172.21.15.1:6823/34863', 'heartbeat_back_addr': '172.21.15.1:6831/34863', 'heartbeat_front_addr': '172.21.15.1:6828/34863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '8a6b018a-3a48-4a01-8526-a386f62ecb36', 'up': 1, 'in': 1, 'weight': 0.198638916015625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 210, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6816', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6818', 'nonce': 34864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6820', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6822', 'nonce': 34864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6827', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6829', 'nonce': 34864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6824', 'nonce': 34864}, {'type': 'v1', 'addr': '172.21.15.1:6825', 'nonce': 34864}]}, 'public_addr': '172.21.15.1:6818/34864', 'cluster_addr': '172.21.15.1:6822/34864', 'heartbeat_back_addr': '172.21.15.1:6829/34864', 'heartbeat_front_addr': '172.21.15.1:6825/34864', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ff089622-e642-4bd0-901c-60efdb87a549', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 208, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6801', 'nonce': 34829}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6803', 'nonce': 34829}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6807', 'nonce': 34829}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 34829}, {'type': 'v1', 'addr': '172.21.15.136:6805', 'nonce': 34829}]}, 'public_addr': '172.21.15.136:6801/34829', 'cluster_addr': '172.21.15.136:6803/34829', 'heartbeat_back_addr': '172.21.15.136:6807/34829', 'heartbeat_front_addr': '172.21.15.136:6805/34829', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3c57c228-dfaa-4e33-bded-27da0048d141', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 211, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6811', 'nonce': 34830}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6815', 'nonce': 34830}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6821', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6823', 'nonce': 34830}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6817', 'nonce': 34830}, {'type': 'v1', 'addr': '172.21.15.136:6819', 'nonce': 34830}]}, 'public_addr': '172.21.15.136:6811/34830', 'cluster_addr': '172.21.15.136:6815/34830', 'heartbeat_back_addr': '172.21.15.136:6823/34830', 'heartbeat_front_addr': '172.21.15.136:6819/34830', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '201f889f-dac1-4a26-a285-7a0ec21c3646', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 94, 'up_from': 132, 'up_thru': 92, 'down_at': 95, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6810', 'nonce': 48096}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6814', 'nonce': 48096}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6820', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6822', 'nonce': 48096}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6816', 'nonce': 48096}, {'type': 'v1', 'addr': '172.21.15.136:6818', 'nonce': 48096}]}, 'public_addr': '172.21.15.136:6810/48096', 'cluster_addr': '172.21.15.136:6814/48096', 'heartbeat_back_addr': '172.21.15.136:6822/48096', 'heartbeat_front_addr': '172.21.15.136:6818/48096', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '79ffd413-a9d3-4180-9691-d834d89c3f2e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 211, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6824', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6825', 'nonce': 34874}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6826', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6827', 'nonce': 34874}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6830', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6831', 'nonce': 34874}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6828', 'nonce': 34874}, {'type': 'v1', 'addr': '172.21.15.136:6829', 'nonce': 34874}]}, 'public_addr': '172.21.15.136:6825/34874', 'cluster_addr': '172.21.15.136:6827/34874', 'heartbeat_back_addr': '172.21.15.136:6831/34874', 'heartbeat_front_addr': '172.21.15.136:6829/34874', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:49.068217+0000', 'dead_epoch': 177}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:09:49.114373+0000', 'dead_epoch': 159}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:10:51.570589+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:10:50.239826+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:10:50.335135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:10:51.341578+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 65536, 'last_purged_snaps_scrub': '2021-01-15T19:11:01.438173+0000', 'dead_epoch': 95}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138297136906239, 'old_weight': 0, 'last_purged_snaps_scrub': '2021-01-15T19:10:51.328325+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '1.0', 'osds': [5, 2]}, {'pgid': '1.d', 'osds': [2, 4]}, {'pgid': '1.14', 'osds': [5, 7]}, {'pgid': '2.a', 'osds': [5, 2]}, {'pgid': '2.c', 'osds': [5, 7]}, {'pgid': '2.d', 'osds': [5, 7]}, {'pgid': '2.13', 'osds': [7, 2]}, {'pgid': '2.1a', 'osds': [5, 2]}, {'pgid': '2.1b', 'osds': [5, 2]}], 'primary_temp': [], 'blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'isaprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '2', 'm': '1', 'name': 'isaprofile', 'plugin': 'isa', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}, {'begin': 38, 'length': 1}, {'begin': 42, 'length': 1}, {'begin': 47, 'length': 1}, {'begin': 56, 'length': 1}, {'begin': 58, 'length': 1}, {'begin': 62, 'length': 1}, {'begin': 65, 'length': 1}, {'begin': 68, 'length': 1}, {'begin': 70, 'length': 1}, {'begin': 75, 'length': 2}, {'begin': 80, 'length': 1}, {'begin': 83, 'length': 2}, {'begin': 86, 'length': 4}, {'begin': 92, 'length': 2}, {'begin': 95, 'length': 2}, {'begin': 98, 'length': 2}, {'begin': 102, 'length': 1}, {'begin': 109, 'length': 2}, {'begin': 112, 'length': 4}, {'begin': 120, 'length': 2}, {'begin': 125, 'length': 1}, {'begin': 132, 'length': 1}, {'begin': 141, 'length': 1}, {'begin': 143, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2021-01-15T19:11:40.886 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2021-01-15T19:11:40.983 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:40.981+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:41.047 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:41.048 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:41.057 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:41.106 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:41.105+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:41.183 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:41.184 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:41.184 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:41.182+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:41.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:41.193 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:41.284 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:41.283+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:41.341 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:41.361 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:41.362 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:41.362 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:41.372 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:41.382 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:41.403 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:41.413 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:41.412+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:41.424 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:41.446 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:41.449 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:41.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:41.487 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:41.486+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:41.538 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:41.539 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:41.539 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:41.540 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:41.540 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:41.549 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:41.590 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:41.589+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:41.689 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:41.689+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:41.716 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:41.716 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:244 3.0s0 3:095f0b32:::smithi00140323-47:head [delete] snapc c=[c,7,3] ondisk+write+peerstat_old+known_if_redirected e36)", 2021-01-15T19:11:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:41.717 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 197.427471414, 2021-01-15T19:11:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999, 2021-01-15T19:11:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.718 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.719 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 244 2021-01-15T19:11:41.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.719 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:41.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286035+0000", 2021-01-15T19:11:41.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7338000000000001e-05 2021-01-15T19:11:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286072+0000", 2021-01-15T19:11:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1540000000000002e-06 2021-01-15T19:11:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286080+0000", 2021-01-15T19:11:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5211999999999999e-05 2021-01-15T19:11:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286095+0000", 2021-01-15T19:11:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2021-01-15T19:11:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286102+0000", 2021-01-15T19:11:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6069999999999993e-05 2021-01-15T19:11:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286199+0000", 2021-01-15T19:11:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1909999999999999e-05 2021-01-15T19:11:41.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.286210+0000", 2021-01-15T19:11:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053930580000000001 2021-01-15T19:11:41.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291604+0000", 2021-01-15T19:11:41.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010724499999999999 2021-01-15T19:11:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.291711+0000", 2021-01-15T19:11:41.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00096169099999999998 2021-01-15T19:11:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.292672+0000", 2021-01-15T19:11:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058538059999999996 2021-01-15T19:11:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298526+0000", 2021-01-15T19:11:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0906999999999998e-05 2021-01-15T19:11:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.298587+0000", 2021-01-15T19:11:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087245599999999997 2021-01-15T19:11:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:24.299460+0000", 2021-01-15T19:11:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013424999999999999 2021-01-15T19:11:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:332 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~425984] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 192.495708648, 2021-01-15T19:11:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004, 2021-01-15T19:11:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 332 2021-01-15T19:11:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.217797+0000", 2021-01-15T19:11:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068652000000000003 2021-01-15T19:11:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218484+0000", 2021-01-15T19:11:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0066e-05 2021-01-15T19:11:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218494+0000", 2021-01-15T19:11:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6949999999999999e-05 2021-01-15T19:11:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218511+0000", 2021-01-15T19:11:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2300000000000002e-06 2021-01-15T19:11:41.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218518+0000", 2021-01-15T19:11:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2848999999999994e-05 2021-01-15T19:11:41.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218581+0000", 2021-01-15T19:11:41.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669000000000001e-05 2021-01-15T19:11:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.218592+0000", 2021-01-15T19:11:41.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783696 2021-01-15T19:11:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220375+0000", 2021-01-15T19:11:41.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010862900000000001 2021-01-15T19:11:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220484+0000", 2021-01-15T19:11:41.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088608299999999997 2021-01-15T19:11:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.221370+0000", 2021-01-15T19:11:41.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031583150000000001 2021-01-15T19:11:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224528+0000", 2021-01-15T19:11:41.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026650110000000001 2021-01-15T19:11:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227193+0000", 2021-01-15T19:11:41.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6819999999999999e-05 2021-01-15T19:11:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227280+0000", 2021-01-15T19:11:41.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094828380000000004 2021-01-15T19:11:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:333 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~409600] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 192.49342060699999, 2021-01-15T19:11:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001, 2021-01-15T19:11:41.755 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.756 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2021-01-15T19:11:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.220085+0000", 2021-01-15T19:11:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025072060000000001 2021-01-15T19:11:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222593+0000", 2021-01-15T19:11:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4800000000000007e-06 2021-01-15T19:11:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222602+0000", 2021-01-15T19:11:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6843000000000001e-05 2021-01-15T19:11:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222619+0000", 2021-01-15T19:11:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1249999999999993e-06 2021-01-15T19:11:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222627+0000", 2021-01-15T19:11:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6610000000000003e-05 2021-01-15T19:11:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222674+0000", 2021-01-15T19:11:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116416 2021-01-15T19:11:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222790+0000", 2021-01-15T19:11:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068520500000000004 2021-01-15T19:11:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223475+0000", 2021-01-15T19:11:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039278389999999998 2021-01-15T19:11:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227403+0000", 2021-01-15T19:11:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042504090000000001 2021-01-15T19:11:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231654+0000", 2021-01-15T19:11:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.831e-05 2021-01-15T19:11:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231742+0000", 2021-01-15T19:11:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011656443000000001 2021-01-15T19:11:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:334 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~393216] snapc 11=[11,10] ondisk+write+known_if_redirected e41)", 2021-01-15T19:11:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 192.49083098200001, 2021-01-15T19:11:41.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993, 2021-01-15T19:11:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.771 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 334 2021-01-15T19:11:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.222675+0000", 2021-01-15T19:11:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073902899999999999 2021-01-15T19:11:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223414+0000", 2021-01-15T19:11:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3960000000000006e-06 2021-01-15T19:11:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223422+0000", 2021-01-15T19:11:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4541e-05 2021-01-15T19:11:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223437+0000", 2021-01-15T19:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9219999999999997e-06 2021-01-15T19:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223445+0000", 2021-01-15T19:11:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010984059999999999 2021-01-15T19:11:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224543+0000", 2021-01-15T19:11:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0479000000000002e-05 2021-01-15T19:11:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.224604+0000", 2021-01-15T19:11:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060414599999999996 2021-01-15T19:11:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.225208+0000", 2021-01-15T19:11:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021981100000000001 2021-01-15T19:11:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227406+0000", 2021-01-15T19:11:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045066769999999997 2021-01-15T19:11:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.231913+0000", 2021-01-15T19:11:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5527999999999997e-05 2021-01-15T19:11:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232008+0000", 2021-01-15T19:11:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093332339999999993 2021-01-15T19:11:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:336 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e41)", 2021-01-15T19:11:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 192.48992270400001, 2021-01-15T19:11:41.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001, 2021-01-15T19:11:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.786 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 336 2021-01-15T19:11:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223583+0000", 2021-01-15T19:11:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8792999999999999e-05 2021-01-15T19:11:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223622+0000", 2021-01-15T19:11:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1630000000000001e-06 2021-01-15T19:11:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223628+0000", 2021-01-15T19:11:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2646999999999999e-05 2021-01-15T19:11:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223641+0000", 2021-01-15T19:11:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2500000000000003e-06 2021-01-15T19:11:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.223647+0000", 2021-01-15T19:11:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037760739999999999 2021-01-15T19:11:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227423+0000", 2021-01-15T19:11:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5331000000000002e-05 2021-01-15T19:11:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.227489+0000", 2021-01-15T19:11:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049474269999999999 2021-01-15T19:11:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232436+0000", 2021-01-15T19:11:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4380000000000001e-05 2021-01-15T19:11:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.232530+0000", 2021-01-15T19:11:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001675239 2021-01-15T19:11:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:29.234206+0000", 2021-01-15T19:11:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010622304000000001 2021-01-15T19:11:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:370 3.0s0 3:0c0290e4:::smithi00140323-22:head [copy-from ver 4] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:41.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.521238797, 2021-01-15T19:11:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002, 2021-01-15T19:11:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.800 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 370 2021-01-15T19:11:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192267+0000", 2021-01-15T19:11:41.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8516e-05 2021-01-15T19:11:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192326+0000", 2021-01-15T19:11:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1331999999999999e-05 2021-01-15T19:11:41.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192337+0000", 2021-01-15T19:11:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8991000000000002e-05 2021-01-15T19:11:41.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192356+0000", 2021-01-15T19:11:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2021-01-15T19:11:41.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192365+0000", 2021-01-15T19:11:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6354000000000003e-05 2021-01-15T19:11:41.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192421+0000", 2021-01-15T19:11:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0519e-05 2021-01-15T19:11:41.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192432+0000", 2021-01-15T19:11:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025807149999999999 2021-01-15T19:11:41.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195013+0000", 2021-01-15T19:11:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110213 2021-01-15T19:11:41.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195123+0000", 2021-01-15T19:11:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071879459999999997 2021-01-15T19:11:41.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.202311+0000", 2021-01-15T19:11:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010676134 2021-01-15T19:11:41.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212987+0000", 2021-01-15T19:11:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1469000000000004e-05 2021-01-15T19:11:41.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213048+0000", 2021-01-15T19:11:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7444e-05 2021-01-15T19:11:41.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213116+0000", 2021-01-15T19:11:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020848686000000002 2021-01-15T19:11:41.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:371 3.0s0 3:0c0290e4:::smithi00140323-22:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.52108906000001, 2021-01-15T19:11:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001, 2021-01-15T19:11:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.817 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 371 2021-01-15T19:11:41.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:41.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192417+0000", 2021-01-15T19:11:41.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5151000000000003e-05 2021-01-15T19:11:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192452+0000", 2021-01-15T19:11:41.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.742e-06 2021-01-15T19:11:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192459+0000", 2021-01-15T19:11:41.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2249000000000001e-05 2021-01-15T19:11:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192471+0000", 2021-01-15T19:11:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3410000000000002e-06 2021-01-15T19:11:41.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192476+0000", 2021-01-15T19:11:41.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3420000000000001e-05 2021-01-15T19:11:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192520+0000", 2021-01-15T19:11:41.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9150000000000002e-06 2021-01-15T19:11:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:41.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192529+0000", 2021-01-15T19:11:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028387799999999999 2021-01-15T19:11:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195368+0000", 2021-01-15T19:11:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5715000000000003e-05 2021-01-15T19:11:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195443+0000", 2021-01-15T19:11:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017403047000000001 2021-01-15T19:11:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212846+0000", 2021-01-15T19:11:41.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3013e-05 2021-01-15T19:11:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212919+0000", 2021-01-15T19:11:41.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002143 2021-01-15T19:11:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213134+0000", 2021-01-15T19:11:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1420000000000005e-05 2021-01-15T19:11:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213195+0000", 2021-01-15T19:11:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2268000000000006e-05 2021-01-15T19:11:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.213277+0000", 2021-01-15T19:11:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020860361000000001 2021-01-15T19:11:41.835 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.837 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:372 3.0s0 3:095f0b32:::smithi00140323-47:head [copy-from ver 6] snapc 15=[15,14,12,10] ondisk+write+known_if_redirected e45)", 2021-01-15T19:11:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:41.838 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.520937162, 2021-01-15T19:11:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999, 2021-01-15T19:11:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.839 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.840 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 372 2021-01-15T19:11:41.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:41.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192569+0000", 2021-01-15T19:11:41.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5884000000000002e-05 2021-01-15T19:11:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192625+0000", 2021-01-15T19:11:41.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3139999999999998e-06 2021-01-15T19:11:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192632+0000", 2021-01-15T19:11:41.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2913e-05 2021-01-15T19:11:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192645+0000", 2021-01-15T19:11:41.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000001e-06 2021-01-15T19:11:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192650+0000", 2021-01-15T19:11:41.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9279e-05 2021-01-15T19:11:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192699+0000", 2021-01-15T19:11:41.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4790000000000002e-06 2021-01-15T19:11:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192709+0000", 2021-01-15T19:11:41.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002753742 2021-01-15T19:11:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195462+0000", 2021-01-15T19:11:41.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8483000000000001e-05 2021-01-15T19:11:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195531+0000", 2021-01-15T19:11:41.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010075835999999999 2021-01-15T19:11:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.205607+0000", 2021-01-15T19:11:41.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011946434000000001 2021-01-15T19:11:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217553+0000", 2021-01-15T19:11:41.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4114e-05 2021-01-15T19:11:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.217647+0000", 2021-01-15T19:11:41.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024363879999999998 2021-01-15T19:11:41.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220084+0000", 2021-01-15T19:11:41.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027514793999999999 2021-01-15T19:11:41.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.870 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.872 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:373 3.0s0 3:095f0b32:::smithi00140323-47:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e45)", 2021-01-15T19:11:41.872 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:41.872 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 188.52081185, 2021-01-15T19:11:41.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999, 2021-01-15T19:11:41.873 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.874 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:41.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.875 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.875 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 373 2021-01-15T19:11:41.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.876 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:41.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192694+0000", 2021-01-15T19:11:41.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8095e-05 2021-01-15T19:11:41.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192732+0000", 2021-01-15T19:11:41.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5540000000000003e-06 2021-01-15T19:11:41.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192739+0000", 2021-01-15T19:11:41.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2576e-05 2021-01-15T19:11:41.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192751+0000", 2021-01-15T19:11:41.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7619999999999998e-06 2021-01-15T19:11:41.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192758+0000", 2021-01-15T19:11:41.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019271699999999999 2021-01-15T19:11:41.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192951+0000", 2021-01-15T19:11:41.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2021-01-15T19:11:41.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:41.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.192959+0000", 2021-01-15T19:11:41.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029529589999999998 2021-01-15T19:11:41.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195912+0000", 2021-01-15T19:11:41.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8649e-05 2021-01-15T19:11:41.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:41.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.195971+0000", 2021-01-15T19:11:41.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016694761999999998 2021-01-15T19:11:41.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212665+0000", 2021-01-15T19:11:41.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132936 2021-01-15T19:11:41.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:41.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.212798+0000", 2021-01-15T19:11:41.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073373090000000002 2021-01-15T19:11:41.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220136+0000", 2021-01-15T19:11:41.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108415 2021-01-15T19:11:41.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220244+0000", 2021-01-15T19:11:41.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012517100000000001 2021-01-15T19:11:41.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:33.220369+0000", 2021-01-15T19:11:41.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027674906999999999 2021-01-15T19:11:41.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.903 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.904 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:379 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~491520] snapc 16=[15,14,12,10] ondisk+write+known_if_redirected e47)", 2021-01-15T19:11:41.904 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:41.904 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 186.49997927499999, 2021-01-15T19:11:41.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114, 2021-01-15T19:11:41.905 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.905 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.905 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 379 2021-01-15T19:11:41.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.906 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:41.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.213527+0000", 2021-01-15T19:11:41.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068570600000000001 2021-01-15T19:11:41.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214212+0000", 2021-01-15T19:11:41.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1070000000000001e-05 2021-01-15T19:11:41.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214224+0000", 2021-01-15T19:11:41.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1043000000000002e-05 2021-01-15T19:11:41.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214245+0000", 2021-01-15T19:11:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0146000000000001e-05 2021-01-15T19:11:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214255+0000", 2021-01-15T19:11:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4710000000000003e-05 2021-01-15T19:11:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214309+0000", 2021-01-15T19:11:41.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013488399999999999 2021-01-15T19:11:41.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.214444+0000", 2021-01-15T19:11:41.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010000969999999999 2021-01-15T19:11:41.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.215444+0000", 2021-01-15T19:11:41.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080230719999999991 2021-01-15T19:11:41.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223467+0000", 2021-01-15T19:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1729000000000005e-05 2021-01-15T19:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223529+0000", 2021-01-15T19:11:41.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7657000000000004e-05 2021-01-15T19:11:41.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223627+0000", 2021-01-15T19:11:41.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010100114 2021-01-15T19:11:41.918 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.918 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.918 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:383 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e47)", 2021-01-15T19:11:41.919 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:41.919 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 186.493685943, 2021-01-15T19:11:41.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149, 2021-01-15T19:11:41.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:41.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.920 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.921 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 383 2021-01-15T19:11:41.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:41.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219820+0000", 2021-01-15T19:11:41.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9945e-05 2021-01-15T19:11:41.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219840+0000", 2021-01-15T19:11:41.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.569e-06 2021-01-15T19:11:41.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219844+0000", 2021-01-15T19:11:41.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3669999999999999e-06 2021-01-15T19:11:41.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219851+0000", 2021-01-15T19:11:41.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5830000000000001e-06 2021-01-15T19:11:41.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.219854+0000", 2021-01-15T19:11:41.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003815868 2021-01-15T19:11:41.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223669+0000", 2021-01-15T19:11:41.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011843 2021-01-15T19:11:41.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:41.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.223788+0000", 2021-01-15T19:11:41.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032150450000000001 2021-01-15T19:11:41.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227003+0000", 2021-01-15T19:11:41.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9144000000000001e-05 2021-01-15T19:11:41.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.227072+0000", 2021-01-15T19:11:41.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046861979999999999 2021-01-15T19:11:41.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:35.231758+0000", 2021-01-15T19:11:41.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011938149 2021-01-15T19:11:41.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.932 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:419 3.0s0 3:007f0442:::smithi00140323-3:head [delete] snapc 1b=[1b,19,14,10] ondisk+write+peerstat_old+known_if_redirected e51)", 2021-01-15T19:11:41.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:41.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.53162642999999, 2021-01-15T19:11:41.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997, 2021-01-15T19:11:41.934 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.934 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.935 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.935 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 419 2021-01-15T19:11:41.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.935 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:41.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181880+0000", 2021-01-15T19:11:41.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5087999999999999e-05 2021-01-15T19:11:41.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181925+0000", 2021-01-15T19:11:41.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2910000000000002e-06 2021-01-15T19:11:41.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181932+0000", 2021-01-15T19:11:41.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4630000000000001e-05 2021-01-15T19:11:41.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181947+0000", 2021-01-15T19:11:41.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9729999999999997e-06 2021-01-15T19:11:41.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.181955+0000", 2021-01-15T19:11:41.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0878000000000002e-05 2021-01-15T19:11:41.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182015+0000", 2021-01-15T19:11:41.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9279999999999999e-06 2021-01-15T19:11:41.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:41.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.182024+0000", 2021-01-15T19:11:41.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016958019999999999 2021-01-15T19:11:41.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183720+0000", 2021-01-15T19:11:41.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102438 2021-01-15T19:11:41.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.183823+0000", 2021-01-15T19:11:41.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063607799999999999 2021-01-15T19:11:41.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184459+0000", 2021-01-15T19:11:41.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019091900000000001 2021-01-15T19:11:41.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.186368+0000", 2021-01-15T19:11:41.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030794373 2021-01-15T19:11:41.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217162+0000", 2021-01-15T19:11:41.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4547000000000003e-05 2021-01-15T19:11:41.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.217237+0000", 2021-01-15T19:11:41.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035357215999999997 2021-01-15T19:11:41.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.949 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.950 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:427 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~647168] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:41.950 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:41.950 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.46925444999999, 2021-01-15T19:11:41.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999, 2021-01-15T19:11:41.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.951 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.952 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.952 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 427 2021-01-15T19:11:41.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:41.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.244252+0000", 2021-01-15T19:11:41.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014143109999999999 2021-01-15T19:11:41.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245666+0000", 2021-01-15T19:11:41.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2960000000000004e-06 2021-01-15T19:11:41.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245672+0000", 2021-01-15T19:11:41.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3458999999999999e-05 2021-01-15T19:11:41.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245686+0000", 2021-01-15T19:11:41.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6630000000000004e-06 2021-01-15T19:11:41.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245692+0000", 2021-01-15T19:11:41.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0018000000000005e-05 2021-01-15T19:11:41.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245772+0000", 2021-01-15T19:11:41.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101445 2021-01-15T19:11:41.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.245874+0000", 2021-01-15T19:11:41.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085125100000000003 2021-01-15T19:11:41.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.246725+0000", 2021-01-15T19:11:41.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007114642 2021-01-15T19:11:41.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253840+0000", 2021-01-15T19:11:41.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072409099999999995 2021-01-15T19:11:41.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254564+0000", 2021-01-15T19:11:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2529999999999999e-05 2021-01-15T19:11:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254626+0000", 2021-01-15T19:11:41.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010374705999999999 2021-01-15T19:11:41.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.964 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.965 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:428 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~606208] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:41.965 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:41.966 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.46595194899999, 2021-01-15T19:11:41.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001, 2021-01-15T19:11:41.966 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.966 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.966 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.967 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.967 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 428 2021-01-15T19:11:41.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:41.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.247554+0000", 2021-01-15T19:11:41.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010754060000000001 2021-01-15T19:11:41.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248629+0000", 2021-01-15T19:11:41.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:41.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248639+0000", 2021-01-15T19:11:41.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5804999999999999e-05 2021-01-15T19:11:41.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248655+0000", 2021-01-15T19:11:41.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8550000000000002e-06 2021-01-15T19:11:41.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248662+0000", 2021-01-15T19:11:41.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031541 2021-01-15T19:11:41.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249694+0000", 2021-01-15T19:11:41.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0852000000000004e-05 2021-01-15T19:11:41.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249775+0000", 2021-01-15T19:11:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068372599999999995 2021-01-15T19:11:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.250459+0000", 2021-01-15T19:11:41.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034269600000000002 2021-01-15T19:11:41.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:41.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039263809999999996 2021-01-15T19:11:41.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257812+0000", 2021-01-15T19:11:41.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9712000000000001e-05 2021-01-15T19:11:41.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257862+0000", 2021-01-15T19:11:41.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010307546000000001 2021-01-15T19:11:41.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.980 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.980 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:429 3.0s0 3:095f0b32:::smithi00140323-47:head [append 0~196608] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:41.980 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:41.981 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.464630083, 2021-01-15T19:11:41.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615, 2021-01-15T19:11:41.981 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.981 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.981 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.982 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 429 2021-01-15T19:11:41.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.982 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:41.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.248876+0000", 2021-01-15T19:11:41.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029597599999999999 2021-01-15T19:11:41.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:41.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249172+0000", 2021-01-15T19:11:41.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0169999999999996e-06 2021-01-15T19:11:41.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:41.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249180+0000", 2021-01-15T19:11:41.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3851e-05 2021-01-15T19:11:41.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:41.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249194+0000", 2021-01-15T19:11:41.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9470000000000003e-06 2021-01-15T19:11:41.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:41.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249202+0000", 2021-01-15T19:11:41.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026105619999999999 2021-01-15T19:11:41.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:41.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251812+0000", 2021-01-15T19:11:41.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014213799999999999 2021-01-15T19:11:41.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:41.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.251954+0000", 2021-01-15T19:11:41.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042407499999999999 2021-01-15T19:11:41.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:41.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.252379+0000", 2021-01-15T19:11:41.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015079640000000001 2021-01-15T19:11:41.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:41.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253886+0000", 2021-01-15T19:11:41.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040843279999999999 2021-01-15T19:11:41.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:41.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.257971+0000", 2021-01-15T19:11:41.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9756999999999998e-05 2021-01-15T19:11:41.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:41.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258021+0000", 2021-01-15T19:11:41.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009144615 2021-01-15T19:11:41.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:41.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:41.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.995 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:430 3.0s0 3:095f0b32:::smithi00140323-47:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:41.995 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:41.996 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.46425983, 2021-01-15T19:11:41.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997, 2021-01-15T19:11:41.996 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:41.996 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:41.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:41.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:41.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:41.997 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 430 2021-01-15T19:11:41.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.997 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:41.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:41.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:41.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:41.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:41.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:41.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:41.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249246+0000", 2021-01-15T19:11:41.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4521000000000002e-05 2021-01-15T19:11:41.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249301+0000", 2021-01-15T19:11:42.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6390000000000003e-06 2021-01-15T19:11:42.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249307+0000", 2021-01-15T19:11:42.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019381800000000001 2021-01-15T19:11:42.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249501+0000", 2021-01-15T19:11:42.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.48e-06 2021-01-15T19:11:42.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249507+0000", 2021-01-15T19:11:42.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035463280000000001 2021-01-15T19:11:42.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253053+0000", 2021-01-15T19:11:42.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9712999999999999e-05 2021-01-15T19:11:42.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253153+0000", 2021-01-15T19:11:42.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047694499999999999 2021-01-15T19:11:42.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.253630+0000", 2021-01-15T19:11:42.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085817100000000002 2021-01-15T19:11:42.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254488+0000", 2021-01-15T19:11:42.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036485950000000001 2021-01-15T19:11:42.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258136+0000", 2021-01-15T19:11:42.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8632999999999999e-05 2021-01-15T19:11:42.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258195+0000", 2021-01-15T19:11:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089488429999999997 2021-01-15T19:11:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.010 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.010 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:431 3.0s0 3:095f0b32:::smithi00140323-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:42.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:42.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 182.463925437, 2021-01-15T19:11:42.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992, 2021-01-15T19:11:42.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.011 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.012 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.012 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 431 2021-01-15T19:11:42.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.012 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:42.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249581+0000", 2021-01-15T19:11:42.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3310000000000001e-05 2021-01-15T19:11:42.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249654+0000", 2021-01-15T19:11:42.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4440000000000002e-06 2021-01-15T19:11:42.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249661+0000", 2021-01-15T19:11:42.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3181e-05 2021-01-15T19:11:42.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249675+0000", 2021-01-15T19:11:42.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3970000000000002e-06 2021-01-15T19:11:42.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.249680+0000", 2021-01-15T19:11:42.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004702393 2021-01-15T19:11:42.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254382+0000", 2021-01-15T19:11:42.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5456000000000005e-05 2021-01-15T19:11:42.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.254448+0000", 2021-01-15T19:11:42.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040797960000000001 2021-01-15T19:11:42.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258528+0000", 2021-01-15T19:11:42.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120104 2021-01-15T19:11:42.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.258648+0000", 2021-01-15T19:11:42.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088150100000000003 2021-01-15T19:11:42.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.259529+0000", 2021-01-15T19:11:42.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099485819999999992 2021-01-15T19:11:42.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.024 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.024 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.13:2 3.18s0 3:1c18e8c7:::smithi00140323-46:head [assert-version v10,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e65)", 2021-01-15T19:11:42.025 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:42.025 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 168.52266417800001, 2021-01-15T19:11:42.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118, 2021-01-15T19:11:42.025 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.025 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.026 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:42.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190842+0000", 2021-01-15T19:11:42.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5757000000000002e-05 2021-01-15T19:11:42.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190898+0000", 2021-01-15T19:11:42.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1500000000000005e-06 2021-01-15T19:11:42.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190907+0000", 2021-01-15T19:11:42.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7088999999999999e-05 2021-01-15T19:11:42.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190924+0000", 2021-01-15T19:11:42.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4190000000000002e-06 2021-01-15T19:11:42.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190932+0000", 2021-01-15T19:11:42.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5917999999999997e-05 2021-01-15T19:11:42.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190968+0000", 2021-01-15T19:11:42.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3910000000000003e-06 2021-01-15T19:11:42.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:42.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.190975+0000", 2021-01-15T19:11:42.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033681190000000001 2021-01-15T19:11:42.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194343+0000", 2021-01-15T19:11:42.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020001599999999999 2021-01-15T19:11:42.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.194543+0000", 2021-01-15T19:11:42.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005334259 2021-01-15T19:11:42.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:53.199877+0000", 2021-01-15T19:11:42.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.009035118 2021-01-15T19:11:42.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.037 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.037 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:745 3.18s0 3:1c18e8c7:::smithi00140323-46:head [stat,delete] snapc 32=[32,2f,2d,2a,28,26,25,22,1f,1b,19] ondisk+write+known_if_redirected e75)", 2021-01-15T19:11:42.038 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:42.038 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.53030130600001, 2021-01-15T19:11:42.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999, 2021-01-15T19:11:42.038 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.038 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:42.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.039 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.039 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 745 2021-01-15T19:11:42.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.040 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:42.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183205+0000", 2021-01-15T19:11:42.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1591000000000002e-05 2021-01-15T19:11:42.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183246+0000", 2021-01-15T19:11:42.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0043e-05 2021-01-15T19:11:42.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183256+0000", 2021-01-15T19:11:42.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357999999999999e-05 2021-01-15T19:11:42.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183275+0000", 2021-01-15T19:11:42.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2021-01-15T19:11:42.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183283+0000", 2021-01-15T19:11:42.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2325999999999999e-05 2021-01-15T19:11:42.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183346+0000", 2021-01-15T19:11:42.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2092999999999999e-05 2021-01-15T19:11:42.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:42.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.183358+0000", 2021-01-15T19:11:42.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00652828 2021-01-15T19:11:42.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.189886+0000", 2021-01-15T19:11:42.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013606100000000001 2021-01-15T19:11:42.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.190022+0000", 2021-01-15T19:11:42.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011961090000000001 2021-01-15T19:11:42.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.191218+0000", 2021-01-15T19:11:42.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037405910000000001 2021-01-15T19:11:42.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.194959+0000", 2021-01-15T19:11:42.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021132009999999999 2021-01-15T19:11:42.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197072+0000", 2021-01-15T19:11:42.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2874000000000006e-05 2021-01-15T19:11:42.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.197155+0000", 2021-01-15T19:11:42.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013950133999999999 2021-01-15T19:11:42.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.054 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.055 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:754 3.18s0 3:1c18e8c7:::smithi00140323-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e75)", 2021-01-15T19:11:42.055 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:42.055 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 158.50030710199999, 2021-01-15T19:11:42.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178, 2021-01-15T19:11:42.056 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.056 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.056 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 754 2021-01-15T19:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:42.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213199+0000", 2021-01-15T19:11:42.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5651999999999997e-05 2021-01-15T19:11:42.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213235+0000", 2021-01-15T19:11:42.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2021-01-15T19:11:42.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213241+0000", 2021-01-15T19:11:42.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8989999999999996e-06 2021-01-15T19:11:42.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213251+0000", 2021-01-15T19:11:42.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2021-01-15T19:11:42.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.213254+0000", 2021-01-15T19:11:42.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023249410000000001 2021-01-15T19:11:42.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215579+0000", 2021-01-15T19:11:42.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011187 2021-01-15T19:11:42.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.215691+0000", 2021-01-15T19:11:42.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055891539999999998 2021-01-15T19:11:42.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221280+0000", 2021-01-15T19:11:42.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000532253 2021-01-15T19:11:42.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.221813+0000", 2021-01-15T19:11:42.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028954409999999999 2021-01-15T19:11:42.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:03.224708+0000", 2021-01-15T19:11:42.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011509178 2021-01-15T19:11:42.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.068 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.069 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.069 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:42.070 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:42.070+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:42.071 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:42.172 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:42.171+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:42.234 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2021-01-15T19:11:42.237 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:42.237 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:42.238 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:42.238 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:42.247 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:42.291 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:42.290+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:42.373 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:42.373 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:42.374 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:42.374 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:42.374 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:42.375 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:42.372+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:42.383 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:42.474 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:42.473+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:42.550 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:42.551 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:42.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:42.552 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2021-01-15T19:11:42.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.553 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 51/21 ECSubWrite(tid=77, reqid=client.4418.0:419, at_version=51'56, trim_to=0'0, roll_forward_to=50'54))", 2021-01-15T19:11:42.553 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:42.553 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 183.362850479, 2021-01-15T19:11:42.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997, 2021-01-15T19:11:42.554 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.555 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:42.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184721+0000", 2021-01-15T19:11:42.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5761e-05 2021-01-15T19:11:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184747+0000", 2021-01-15T19:11:42.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9839000000000001e-05 2021-01-15T19:11:42.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184767+0000", 2021-01-15T19:11:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1889999999999996e-06 2021-01-15T19:11:42.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184776+0000", 2021-01-15T19:11:42.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1610000000000002e-06 2021-01-15T19:11:42.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184781+0000", 2021-01-15T19:11:42.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9669000000000002e-05 2021-01-15T19:11:42.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184831+0000", 2021-01-15T19:11:42.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4479999999999994e-06 2021-01-15T19:11:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.184839+0000", 2021-01-15T19:11:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031996942 2021-01-15T19:11:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216836+0000", 2021-01-15T19:11:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6344000000000002e-05 2021-01-15T19:11:42.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:39.216883+0000", 2021-01-15T19:11:42.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032161352999999997 2021-01-15T19:11:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:511 3.bs0 3:db20b9e4:::smithi00140323-44:head [copy-from ver 38] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:42.574 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 179.31536230200001, 2021-01-15T19:11:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001, 2021-01-15T19:11:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:42.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 511 2021-01-15T19:11:42.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.577 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:42.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232209+0000", 2021-01-15T19:11:42.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9429999999999999e-05 2021-01-15T19:11:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232269+0000", 2021-01-15T19:11:42.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0893999999999999e-05 2021-01-15T19:11:42.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232280+0000", 2021-01-15T19:11:42.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9076000000000002e-05 2021-01-15T19:11:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232299+0000", 2021-01-15T19:11:42.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9889999999999992e-06 2021-01-15T19:11:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232307+0000", 2021-01-15T19:11:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6443000000000001e-05 2021-01-15T19:11:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232373+0000", 2021-01-15T19:11:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123385 2021-01-15T19:11:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232497+0000", 2021-01-15T19:11:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015366009999999999 2021-01-15T19:11:42.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.247863+0000", 2021-01-15T19:11:42.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084574579999999993 2021-01-15T19:11:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256320+0000", 2021-01-15T19:11:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023124439999999999 2021-01-15T19:11:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258633+0000", 2021-01-15T19:11:42.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6273999999999995e-05 2021-01-15T19:11:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.258709+0000", 2021-01-15T19:11:42.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026499403000000001 2021-01-15T19:11:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.596 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.598 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:512 3.bs0 3:db20b9e4:::smithi00140323-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e55)", 2021-01-15T19:11:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 179.31521428400001, 2021-01-15T19:11:42.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216, 2021-01-15T19:11:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 512 2021-01-15T19:11:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232357+0000", 2021-01-15T19:11:42.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0658000000000003e-05 2021-01-15T19:11:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232398+0000", 2021-01-15T19:11:42.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2530000000000001e-06 2021-01-15T19:11:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232404+0000", 2021-01-15T19:11:42.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5061e-05 2021-01-15T19:11:42.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232419+0000", 2021-01-15T19:11:42.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9230000000000001e-06 2021-01-15T19:11:42.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232424+0000", 2021-01-15T19:11:42.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029343499999999998 2021-01-15T19:11:42.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232718+0000", 2021-01-15T19:11:42.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1389000000000005e-05 2021-01-15T19:11:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.232779+0000", 2021-01-15T19:11:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023341310000000001 2021-01-15T19:11:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256120+0000", 2021-01-15T19:11:42.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114956 2021-01-15T19:11:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.256235+0000", 2021-01-15T19:11:42.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030425579999999999 2021-01-15T19:11:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259278+0000", 2021-01-15T19:11:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101216 2021-01-15T19:11:42.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259379+0000", 2021-01-15T19:11:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0401000000000004e-05 2021-01-15T19:11:42.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:08:43.259470+0000", 2021-01-15T19:11:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.02711216 2021-01-15T19:11:42.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.625 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:927 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.360893293, 2021-01-15T19:11:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999, 2021-01-15T19:11:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.627 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 927 2021-01-15T19:11:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186678+0000", 2021-01-15T19:11:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7673999999999997e-05 2021-01-15T19:11:42.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186736+0000", 2021-01-15T19:11:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7339999999999997e-06 2021-01-15T19:11:42.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186745+0000", 2021-01-15T19:11:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6095000000000001e-05 2021-01-15T19:11:42.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186761+0000", 2021-01-15T19:11:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4848000000000001e-05 2021-01-15T19:11:42.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186776+0000", 2021-01-15T19:11:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9145999999999999e-05 2021-01-15T19:11:42.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186835+0000", 2021-01-15T19:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4520000000000002e-06 2021-01-15T19:11:42.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186838+0000", 2021-01-15T19:11:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.021158472 2021-01-15T19:11:42.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.207997+0000", 2021-01-15T19:11:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030383500000000002 2021-01-15T19:11:42.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208301+0000", 2021-01-15T19:11:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014600752 2021-01-15T19:11:42.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.222901+0000", 2021-01-15T19:11:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362230079999999 2021-01-15T19:11:42.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:928 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:42.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.36070200399999, 2021-01-15T19:11:42.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001, 2021-01-15T19:11:42.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.643 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 928 2021-01-15T19:11:42.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:42.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186870+0000", 2021-01-15T19:11:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4858999999999999e-05 2021-01-15T19:11:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186915+0000", 2021-01-15T19:11:42.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4429999999999997e-06 2021-01-15T19:11:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186922+0000", 2021-01-15T19:11:42.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3865e-05 2021-01-15T19:11:42.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186936+0000", 2021-01-15T19:11:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7529999999999998e-06 2021-01-15T19:11:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186942+0000", 2021-01-15T19:11:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6193999999999999e-05 2021-01-15T19:11:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186988+0000", 2021-01-15T19:11:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6339999999999999e-06 2021-01-15T19:11:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.186989+0000", 2021-01-15T19:11:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0216434329999999 2021-01-15T19:11:42.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208633+0000", 2021-01-15T19:11:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018447200000000001 2021-01-15T19:11:42.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.208817+0000", 2021-01-15T19:11:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017602880000000001 2021-01-15T19:11:42.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.226420+0000", 2021-01-15T19:11:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0395505330000001 2021-01-15T19:11:42.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.658 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:929 3.0s1 3:007f0442:::smithi00140323-3:head [read 0~2703360,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e90)", 2021-01-15T19:11:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 144.36053965100001, 2021-01-15T19:11:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001, 2021-01-15T19:11:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.659 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 929 2021-01-15T19:11:42.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:42.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187032+0000", 2021-01-15T19:11:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7849000000000003e-05 2021-01-15T19:11:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187070+0000", 2021-01-15T19:11:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6420000000000007e-06 2021-01-15T19:11:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187078+0000", 2021-01-15T19:11:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4071e-05 2021-01-15T19:11:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187092+0000", 2021-01-15T19:11:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6420000000000004e-06 2021-01-15T19:11:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187097+0000", 2021-01-15T19:11:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000131738 2021-01-15T19:11:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187229+0000", 2021-01-15T19:11:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3350000000000001e-06 2021-01-15T19:11:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2021-01-15T19:11:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:18.187231+0000", 2021-01-15T19:11:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0227386700000001 2021-01-15T19:11:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.209970+0000", 2021-01-15T19:11:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177135 2021-01-15T19:11:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.210147+0000", 2021-01-15T19:11:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018942276000000001 2021-01-15T19:11:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:19.229089+0000", 2021-01-15T19:11:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0420573580000001 2021-01-15T19:11:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.672 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:983 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:42.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 138.32881973799999, 2021-01-15T19:11:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002, 2021-01-15T19:11:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 983 2021-01-15T19:11:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218752+0000", 2021-01-15T19:11:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5401999999999998e-05 2021-01-15T19:11:42.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218777+0000", 2021-01-15T19:11:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1870000000000001e-06 2021-01-15T19:11:42.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218781+0000", 2021-01-15T19:11:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.784e-06 2021-01-15T19:11:42.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218786+0000", 2021-01-15T19:11:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2919999999999998e-06 2021-01-15T19:11:42.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218789+0000", 2021-01-15T19:11:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00066736599999999997 2021-01-15T19:11:42.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219456+0000", 2021-01-15T19:11:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012946 2021-01-15T19:11:42.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219586+0000", 2021-01-15T19:11:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017008769999999999 2021-01-15T19:11:42.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.236594+0000", 2021-01-15T19:11:42.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017842261000000002 2021-01-15T19:11:42.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.686 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:984 3.bs0 3:db20b9e4:::smithi00140323-44:head [read 0~2703360,checksum] snapc 0=[] ondisk+read+known_if_redirected e96)", 2021-01-15T19:11:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 138.32875804899999, 2021-01-15T19:11:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998, 2021-01-15T19:11:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 984 2021-01-15T19:11:42.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:42.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218814+0000", 2021-01-15T19:11:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6083000000000001e-05 2021-01-15T19:11:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218840+0000", 2021-01-15T19:11:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6260000000000001e-06 2021-01-15T19:11:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218843+0000", 2021-01-15T19:11:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9950000000000002e-06 2021-01-15T19:11:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218849+0000", 2021-01-15T19:11:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4140000000000001e-06 2021-01-15T19:11:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.218852+0000", 2021-01-15T19:11:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010404730000000001 2021-01-15T19:11:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.219892+0000", 2021-01-15T19:11:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012405900000000001 2021-01-15T19:11:42.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.220016+0000", 2021-01-15T19:11:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018819487999999999 2021-01-15T19:11:42.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:09:24.238836+0000", 2021-01-15T19:11:42.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020022137999999998 2021-01-15T19:11:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1248 3.9s0 3:921b4c50:::smithi00140323-23:head [copy-from ver 140] snapc 67=[67,66,64,63,60,5f,5c,58,57,56,4b,44,26] ondisk+write+known_if_redirected e133)", 2021-01-15T19:11:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:42.700 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.22469171100001, 2021-01-15T19:11:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003, 2021-01-15T19:11:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:42.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1248 2021-01-15T19:11:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.702 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:42.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322880+0000", 2021-01-15T19:11:42.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1099000000000002e-05 2021-01-15T19:11:42.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322911+0000", 2021-01-15T19:11:42.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-06 2021-01-15T19:11:42.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322918+0000", 2021-01-15T19:11:42.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1722000000000001e-05 2021-01-15T19:11:42.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322929+0000", 2021-01-15T19:11:42.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1520000000000003e-06 2021-01-15T19:11:42.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322937+0000", 2021-01-15T19:11:42.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9224000000000002e-05 2021-01-15T19:11:42.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:42.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3319999999999992e-06 2021-01-15T19:11:42.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:42.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322994+0000", 2021-01-15T19:11:42.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023681700000000002 2021-01-15T19:11:42.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325362+0000", 2021-01-15T19:11:42.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142612 2021-01-15T19:11:42.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325505+0000", 2021-01-15T19:11:42.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031063331999999999 2021-01-15T19:11:42.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.356568+0000", 2021-01-15T19:11:42.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091607159999999993 2021-01-15T19:11:42.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365729+0000", 2021-01-15T19:11:42.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877750000000001 2021-01-15T19:11:42.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368317+0000", 2021-01-15T19:11:42.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6045999999999996e-05 2021-01-15T19:11:42.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.368403+0000", 2021-01-15T19:11:42.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.045522720000000003 2021-01-15T19:11:42.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.719 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1249 3.9s0 3:921b4c50:::smithi00140323-23:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e133)", 2021-01-15T19:11:42.719 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:42.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 101.224605139, 2021-01-15T19:11:42.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003, 2021-01-15T19:11:42.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.720 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.720 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1249 2021-01-15T19:11:42.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.721 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:42.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322967+0000", 2021-01-15T19:11:42.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9417000000000001e-05 2021-01-15T19:11:42.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322986+0000", 2021-01-15T19:11:42.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7000000000000002e-06 2021-01-15T19:11:42.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322990+0000", 2021-01-15T19:11:42.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3810000000000001e-06 2021-01-15T19:11:42.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.322997+0000", 2021-01-15T19:11:42.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8210000000000002e-06 2021-01-15T19:11:42.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323000+0000", 2021-01-15T19:11:42.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5899999999999997e-05 2021-01-15T19:11:42.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323056+0000", 2021-01-15T19:11:42.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2796000000000001e-05 2021-01-15T19:11:42.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:42.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.323069+0000", 2021-01-15T19:11:42.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026144359999999999 2021-01-15T19:11:42.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325683+0000", 2021-01-15T19:11:42.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9987999999999995e-05 2021-01-15T19:11:42.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:42.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.325753+0000", 2021-01-15T19:11:42.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.039672371999999997 2021-01-15T19:11:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365425+0000", 2021-01-15T19:11:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102709 2021-01-15T19:11:42.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.365528+0000", 2021-01-15T19:11:42.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004311189 2021-01-15T19:11:42.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369839+0000", 2021-01-15T19:11:42.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123337 2021-01-15T19:11:42.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.369963+0000", 2021-01-15T19:11:42.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015381100000000001 2021-01-15T19:11:42.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:01.370116+0000", 2021-01-15T19:11:42.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047149857000000003 2021-01-15T19:11:42.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.737 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.738 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s1 135/52 ECSubWrite(tid=614, reqid=osd.4.0:614, at_version=135'232, trim_to=0'0, roll_forward_to=134'230))", 2021-01-15T19:11:42.738 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:42.738 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 99.334295010000005, 2021-01-15T19:11:42.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999, 2021-01-15T19:11:42.738 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.739 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2021-01-15T19:11:42.739 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:42.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213277+0000", 2021-01-15T19:11:42.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9885999999999997e-05 2021-01-15T19:11:42.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213317+0000", 2021-01-15T19:11:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6555999999999999e-05 2021-01-15T19:11:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213333+0000", 2021-01-15T19:11:42.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1880000000000002e-06 2021-01-15T19:11:42.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213341+0000", 2021-01-15T19:11:42.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4460000000000001e-06 2021-01-15T19:11:42.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.213345+0000", 2021-01-15T19:11:42.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015624719 2021-01-15T19:11:42.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228970+0000", 2021-01-15T19:11:42.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8299999999999996e-06 2021-01-15T19:11:42.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.228977+0000", 2021-01-15T19:11:42.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043777659999999999 2021-01-15T19:11:42.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233355+0000", 2021-01-15T19:11:42.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9215999999999999e-05 2021-01-15T19:11:42.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:03.233394+0000", 2021-01-15T19:11:42.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020117606999999999 2021-01-15T19:11:42.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1322 3.11s0 3:8f2f8304:::smithi00140323-26:head [copy-from ver 120] snapc 6d=[6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e138)", 2021-01-15T19:11:42.751 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:42.751 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.209134160999994, 2021-01-15T19:11:42.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999, 2021-01-15T19:11:42.751 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.751 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:42.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.752 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1322 2021-01-15T19:11:42.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:42.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338438+0000", 2021-01-15T19:11:42.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7756e-05 2021-01-15T19:11:42.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338465+0000", 2021-01-15T19:11:42.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0192e-05 2021-01-15T19:11:42.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338476+0000", 2021-01-15T19:11:42.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.302e-06 2021-01-15T19:11:42.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338485+0000", 2021-01-15T19:11:42.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1490000000000002e-06 2021-01-15T19:11:42.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338490+0000", 2021-01-15T19:11:42.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3662000000000003e-05 2021-01-15T19:11:42.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:42.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012364400000000001 2021-01-15T19:11:42.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338657+0000", 2021-01-15T19:11:42.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077146430000000002 2021-01-15T19:11:42.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.346372+0000", 2021-01-15T19:11:42.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082292770000000001 2021-01-15T19:11:42.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.354601+0000", 2021-01-15T19:11:42.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025795380000000001 2021-01-15T19:11:42.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357181+0000", 2021-01-15T19:11:42.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8578000000000004e-05 2021-01-15T19:11:42.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.357279+0000", 2021-01-15T19:11:42.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018841740999999999 2021-01-15T19:11:42.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.765 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.766 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1323 3.11s0 3:8f2f8304:::smithi00140323-26:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e138)", 2021-01-15T19:11:42.766 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:42.766 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 96.209038241000002, 2021-01-15T19:11:42.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002, 2021-01-15T19:11:42.767 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.767 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.768 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1323 2021-01-15T19:11:42.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.768 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:42.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338534+0000", 2021-01-15T19:11:42.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6415000000000003e-05 2021-01-15T19:11:42.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338570+0000", 2021-01-15T19:11:42.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.691e-06 2021-01-15T19:11:42.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338577+0000", 2021-01-15T19:11:42.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2438e-05 2021-01-15T19:11:42.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338589+0000", 2021-01-15T19:11:42.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8459999999999999e-06 2021-01-15T19:11:42.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338594+0000", 2021-01-15T19:11:42.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183275 2021-01-15T19:11:42.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338777+0000", 2021-01-15T19:11:42.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7256999999999998e-05 2021-01-15T19:11:42.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:42.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.338824+0000", 2021-01-15T19:11:42.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012856746 2021-01-15T19:11:42.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351681+0000", 2021-01-15T19:11:42.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4036999999999994e-05 2021-01-15T19:11:42.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.351775+0000", 2021-01-15T19:11:42.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083130579999999999 2021-01-15T19:11:42.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360088+0000", 2021-01-15T19:11:42.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1390999999999995e-05 2021-01-15T19:11:42.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360160+0000", 2021-01-15T19:11:42.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4627000000000001e-05 2021-01-15T19:11:42.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:06.360254+0000", 2021-01-15T19:11:42.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021720781000000002 2021-01-15T19:11:42.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.782 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.783 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1390 3.9s0 3:940930a3:::smithi00140323-8:head [copy-from ver 34] snapc 72=[72,71,70,6f,6e,6d,6b,67,66,64,63,60,5f,56,4b,44,26] ondisk+write+known_if_redirected e143)", 2021-01-15T19:11:42.783 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:42.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.341979901000002, 2021-01-15T19:11:42.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997, 2021-01-15T19:11:42.784 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.784 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:42.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.785 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1390 2021-01-15T19:11:42.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.785 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:42.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205592+0000", 2021-01-15T19:11:42.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2280000000000003e-05 2021-01-15T19:11:42.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205624+0000", 2021-01-15T19:11:42.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8639999999999998e-06 2021-01-15T19:11:42.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205631+0000", 2021-01-15T19:11:42.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3101e-05 2021-01-15T19:11:42.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205644+0000", 2021-01-15T19:11:42.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5549999999999997e-06 2021-01-15T19:11:42.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205652+0000", 2021-01-15T19:11:42.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0504999999999998e-05 2021-01-15T19:11:42.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:42.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9540000000000004e-06 2021-01-15T19:11:42.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:42.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205710+0000", 2021-01-15T19:11:42.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001904296 2021-01-15T19:11:42.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207614+0000", 2021-01-15T19:11:42.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016311099999999999 2021-01-15T19:11:42.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207778+0000", 2021-01-15T19:11:42.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016835218999999998 2021-01-15T19:11:42.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.224613+0000", 2021-01-15T19:11:42.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015265526999999999 2021-01-15T19:11:42.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.239878+0000", 2021-01-15T19:11:42.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062154460000000003 2021-01-15T19:11:42.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246094+0000", 2021-01-15T19:11:42.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7398999999999994e-05 2021-01-15T19:11:42.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.246181+0000", 2021-01-15T19:11:42.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040589256999999997 2021-01-15T19:11:42.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.800 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.801 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1391 3.9s0 3:940930a3:::smithi00140323-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e143)", 2021-01-15T19:11:42.801 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:42.801 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.341888823999994, 2021-01-15T19:11:42.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723, 2021-01-15T19:11:42.802 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.802 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.803 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1391 2021-01-15T19:11:42.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.803 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:42.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205683+0000", 2021-01-15T19:11:42.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2021-01-15T19:11:42.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205702+0000", 2021-01-15T19:11:42.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8459999999999996e-06 2021-01-15T19:11:42.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205708+0000", 2021-01-15T19:11:42.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5299999999999999e-06 2021-01-15T19:11:42.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205716+0000", 2021-01-15T19:11:42.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9519999999999999e-06 2021-01-15T19:11:42.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205719+0000", 2021-01-15T19:11:42.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3032999999999999e-05 2021-01-15T19:11:42.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205772+0000", 2021-01-15T19:11:42.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1940000000000001e-06 2021-01-15T19:11:42.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:42.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.205779+0000", 2021-01-15T19:11:42.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002218809 2021-01-15T19:11:42.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.207998+0000", 2021-01-15T19:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6002999999999997e-05 2021-01-15T19:11:42.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:42.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.208074+0000", 2021-01-15T19:11:42.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030492596 2021-01-15T19:11:42.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238566+0000", 2021-01-15T19:11:42.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133454 2021-01-15T19:11:42.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.238700+0000", 2021-01-15T19:11:42.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01020363 2021-01-15T19:11:42.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248903+0000", 2021-01-15T19:11:42.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3984000000000002e-05 2021-01-15T19:11:42.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.248977+0000", 2021-01-15T19:11:42.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159352 2021-01-15T19:11:42.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:11.249137+0000", 2021-01-15T19:11:42.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043453723 2021-01-15T19:11:42.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1875 3.cs1 3:30ee460e:::smithi00140323-45:head [copy-from ver 101] snapc 98=[98,97,96,93,8e,8c,86,83,82,7b,7a,6f,6b,67,64] ondisk+write+known_if_redirected e181)", 2021-01-15T19:11:42.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:42.820 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.332409568999999, 2021-01-15T19:11:42.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002, 2021-01-15T19:11:42.820 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:42.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.821 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1875 2021-01-15T19:11:42.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.822 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:42.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215162+0000", 2021-01-15T19:11:42.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8470000000000001e-05 2021-01-15T19:11:42.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215221+0000", 2021-01-15T19:11:42.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0740999999999999e-05 2021-01-15T19:11:42.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215231+0000", 2021-01-15T19:11:42.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7118000000000001e-05 2021-01-15T19:11:42.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215249+0000", 2021-01-15T19:11:42.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3890000000000005e-06 2021-01-15T19:11:42.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215257+0000", 2021-01-15T19:11:42.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5821e-05 2021-01-15T19:11:42.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215313+0000", 2021-01-15T19:11:42.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.294e-06 2021-01-15T19:11:42.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2021-01-15T19:11:42.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215320+0000", 2021-01-15T19:11:42.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043186359999999998 2021-01-15T19:11:42.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219639+0000", 2021-01-15T19:11:42.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022064 2021-01-15T19:11:42.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.219859+0000", 2021-01-15T19:11:42.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010162731 2021-01-15T19:11:42.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2021-01-15T19:11:42.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.230022+0000", 2021-01-15T19:11:42.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012425123999999999 2021-01-15T19:11:42.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2021-01-15T19:11:42.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.242447+0000", 2021-01-15T19:11:42.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413171 2021-01-15T19:11:42.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2021-01-15T19:11:42.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.243860+0000", 2021-01-15T19:11:42.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023630600000000001 2021-01-15T19:11:42.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.244097+0000", 2021-01-15T19:11:42.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028934441000000002 2021-01-15T19:11:42.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.837 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.837 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1876 3.cs1 3:30ee460e:::smithi00140323-45:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e181)", 2021-01-15T19:11:42.838 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:42.838 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.332264545000001, 2021-01-15T19:11:42.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402, 2021-01-15T19:11:42.838 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.838 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.839 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.839 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.839 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.839 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1876 2021-01-15T19:11:42.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.840 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:42.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215307+0000", 2021-01-15T19:11:42.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0414999999999998e-05 2021-01-15T19:11:42.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215348+0000", 2021-01-15T19:11:42.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6640000000000001e-06 2021-01-15T19:11:42.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215354+0000", 2021-01-15T19:11:42.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2943e-05 2021-01-15T19:11:42.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215367+0000", 2021-01-15T19:11:42.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7700000000000001e-06 2021-01-15T19:11:42.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215372+0000", 2021-01-15T19:11:42.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0212000000000001e-05 2021-01-15T19:11:42.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215412+0000", 2021-01-15T19:11:42.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2450000000000004e-06 2021-01-15T19:11:42.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:42.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.215421+0000", 2021-01-15T19:11:42.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046720750000000004 2021-01-15T19:11:42.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220094+0000", 2021-01-15T19:11:42.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6395000000000003e-05 2021-01-15T19:11:42.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:42.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.220170+0000", 2021-01-15T19:11:42.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020971659 2021-01-15T19:11:42.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241142+0000", 2021-01-15T19:11:42.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148675 2021-01-15T19:11:42.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:42.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.241290+0000", 2021-01-15T19:11:42.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050345379999999999 2021-01-15T19:11:42.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246325+0000", 2021-01-15T19:11:42.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4138000000000001e-05 2021-01-15T19:11:42.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246399+0000", 2021-01-15T19:11:42.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000127673 2021-01-15T19:11:42.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.246527+0000", 2021-01-15T19:11:42.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031219402 2021-01-15T19:11:42.856 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.856 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.856 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.857 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1894 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:42.857 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:42.857 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.166770671999998, 2021-01-15T19:11:42.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998, 2021-01-15T19:11:42.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.859 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1894 2021-01-15T19:11:42.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.859 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:42.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380801+0000", 2021-01-15T19:11:42.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4573000000000003e-05 2021-01-15T19:11:42.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380836+0000", 2021-01-15T19:11:42.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8010000000000001e-06 2021-01-15T19:11:42.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380839+0000", 2021-01-15T19:11:42.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6449999999999999e-06 2021-01-15T19:11:42.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380846+0000", 2021-01-15T19:11:42.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8380000000000002e-06 2021-01-15T19:11:42.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380849+0000", 2021-01-15T19:11:42.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4465000000000001e-05 2021-01-15T19:11:42.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380903+0000", 2021-01-15T19:11:42.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019121200000000001 2021-01-15T19:11:42.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381095+0000", 2021-01-15T19:11:42.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018764796 2021-01-15T19:11:42.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.399859+0000", 2021-01-15T19:11:42.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019058329999999998 2021-01-15T19:11:42.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.869 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.869 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.870 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1895 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:42.870 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:42.870 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.166683245000002, 2021-01-15T19:11:42.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999, 2021-01-15T19:11:42.870 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.871 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1895 2021-01-15T19:11:42.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.872 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:42.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380889+0000", 2021-01-15T19:11:42.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0385000000000001e-05 2021-01-15T19:11:42.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380919+0000", 2021-01-15T19:11:42.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8750000000000002e-06 2021-01-15T19:11:42.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380923+0000", 2021-01-15T19:11:42.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7499999999999997e-06 2021-01-15T19:11:42.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380930+0000", 2021-01-15T19:11:42.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7759999999999999e-06 2021-01-15T19:11:42.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380933+0000", 2021-01-15T19:11:42.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000436876 2021-01-15T19:11:42.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381370+0000", 2021-01-15T19:11:42.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012544100000000001 2021-01-15T19:11:42.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381496+0000", 2021-01-15T19:11:42.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.023775790000000002 2021-01-15T19:11:42.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.405271+0000", 2021-01-15T19:11:42.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.024382892999999999 2021-01-15T19:11:42.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.882 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.882 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.882 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4418.0:1896 3.cs1 3:30718dd9:::smithi00140323-34:7a [read 0~4972544,checksum] snapc 0=[] ondisk+read+known_if_redirected e181)", 2021-01-15T19:11:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.166603281, 2021-01-15T19:11:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999, 2021-01-15T19:11:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2021-01-15T19:11:42.883 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2021-01-15T19:11:42.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2021-01-15T19:11:42.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4418", 2021-01-15T19:11:42.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:42.884 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1896 2021-01-15T19:11:42.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.885 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2021-01-15T19:11:42.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2021-01-15T19:11:42.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:42.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2021-01-15T19:11:42.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2021-01-15T19:11:42.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.380968+0000", 2021-01-15T19:11:42.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5868000000000002e-05 2021-01-15T19:11:42.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2021-01-15T19:11:42.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381014+0000", 2021-01-15T19:11:42.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6760000000000001e-06 2021-01-15T19:11:42.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2021-01-15T19:11:42.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381021+0000", 2021-01-15T19:11:42.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1296e-05 2021-01-15T19:11:42.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2021-01-15T19:11:42.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381032+0000", 2021-01-15T19:11:42.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3749999999999996e-06 2021-01-15T19:11:42.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2021-01-15T19:11:42.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.381037+0000", 2021-01-15T19:11:42.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032231069999999998 2021-01-15T19:11:42.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2021-01-15T19:11:42.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384260+0000", 2021-01-15T19:11:42.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134573 2021-01-15T19:11:42.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2021-01-15T19:11:42.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.384394+0000", 2021-01-15T19:11:42.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025362518000000001 2021-01-15T19:11:42.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2021-01-15T19:11:42.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2021-01-15T19:11:42.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2021-01-15T19:11:42.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2021-01-15T19:10:49.409757+0000", 2021-01-15T19:11:42.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028788412999999999 2021-01-15T19:11:42.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.895 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.895 INFO:teuthology.orchestra.run.smithi136.stdout: } 2021-01-15T19:11:42.895 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2021-01-15T19:11:42.895 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:42.897 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:42.898 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:42.896+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:42.997 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:42.996+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:43.062 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:43.063 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:43.063 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2021-01-15T19:11:43.063 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:43.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:43.113 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:43.112+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:43.199 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:43.198+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:43.240 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:43.240 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:43.241 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:43.241 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:43.250 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:noscrub is set 2021-01-15T19:11:43.335 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:43.334+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:43.402 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:43.401+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:43.416 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:43.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:43.418 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:11:43.418 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:43.426 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:43.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2021-01-15T19:11:43.503 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:43.502+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:43.569 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:43.569 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:43.570 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:43.570 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:43.580 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:43.625 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:43.623+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:43.625 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:43.646 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:43.666 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:43.687 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:43.704 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:43.704+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:43.707 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:43.715 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:43.716 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:43.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:43.730 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'false' 2021-01-15T19:11:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'false' 2021-01-15T19:11:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'false' 2021-01-15T19:11:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'false' 2021-01-15T19:11:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'false' 2021-01-15T19:11:43.732 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'false' 2021-01-15T19:11:43.733 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:43.733 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:43.805 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:43.805+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:43.861 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:43.861 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:43.871 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:43.906 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:43.905+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:44.002 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.003 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.007 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:44.006+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:44.013 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:44.108 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:44.107+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:44.184 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.185 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.198 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:44.208 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:44.207+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:44.309 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:44.309+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:44.331 INFO:teuthology.orchestra.run.smithi001.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.331 INFO:teuthology.orchestra.run.smithi001.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.341 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2021-01-15T19:11:44.411 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:44.409+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:44.478 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.478 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2021-01-15T19:11:44.532 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:44.531+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:44.612 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.612 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.613 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:44.611+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:44.622 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2021-01-15T19:11:44.714 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:44.712+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:44.786 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.787 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.796 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2021-01-15T19:11:44.814 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:44.813+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:44.914 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:44.913+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:44.960 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:44.960 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:44.969 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2021-01-15T19:11:45.016 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:45.015+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:45.094 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:45.094 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:45.103 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2021-01-15T19:11:45.144 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:45.143+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:45.218 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:45.217+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:45.269 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:45.269 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:45.279 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2021-01-15T19:11:45.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2021-01-15T19:11:45.340 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:45.339+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:45.404 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:45.404 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:45.414 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2021-01-15T19:11:45.458 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:45.457+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:45.520 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:45.519+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:45.538 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:45.538 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:45.548 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2021-01-15T19:11:45.620 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:45.619+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:45.712 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:45.713 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:45.722 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:45.721+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:45.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2021-01-15T19:11:45.734 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2021-01-15T19:11:45.822 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:45.821+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:45.847 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2021-01-15T19:11:45.847 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2021-01-15T19:11:45.856 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2021-01-15T19:11:45.882 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2: {} 2021-01-15T19:11:45.887 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7, 5] out_osds: [6] dead_osds: [1, 0] live_osds: [2, 3, 4, 5, 7, 6] 2021-01-15T19:11:45.887 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2021-01-15T19:11:45.887 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 7, 5] 2021-01-15T19:11:45.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2021-01-15T19:11:45.902 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3: {} 2021-01-15T19:11:45.922 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:45.921+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:45.923 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4: {} 2021-01-15T19:11:45.944 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5: {} 2021-01-15T19:11:45.965 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6: {} 2021-01-15T19:11:45.983 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:45.983 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2021-01-15T19:11:45.984 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2021-01-15T19:11:45.984 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2021-01-15T19:11:45.984 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:45.985 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7: {} 2021-01-15T19:11:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.0 2021-01-15T19:11:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:osd.0: problem getting command descriptions from osd.0 2021-01-15T19:11:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:Error ENXIO: problem getting command descriptions from osd.1 2021-01-15T19:11:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:osd.1: problem getting command descriptions from osd.1 2021-01-15T19:11:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:osd.2: osd_enable_op_tracker = 'true' 2021-01-15T19:11:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:osd.3: osd_enable_op_tracker = 'true' 2021-01-15T19:11:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:osd.4: osd_enable_op_tracker = 'true' 2021-01-15T19:11:45.987 INFO:teuthology.orchestra.run.smithi001.stderr:osd.5: osd_enable_op_tracker = 'true' 2021-01-15T19:11:45.987 INFO:teuthology.orchestra.run.smithi001.stderr:osd.6: osd_enable_op_tracker = 'true' 2021-01-15T19:11:45.987 INFO:teuthology.orchestra.run.smithi001.stderr:osd.7: osd_enable_op_tracker = 'true' 2021-01-15T19:11:45.988 DEBUG:teuthology.orchestra.run:got remote process result: 6 2021-01-15T19:11:45.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2021-01-15T19:11:45.994 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2021-01-15T19:11:46.035 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:46.034+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:46.125 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:46.124+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:46.158 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2021-01-15T19:11:46.159 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2021-01-15T19:11:46.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2021-01-15T19:11:46.159 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2021-01-15T19:11:46.159 INFO:teuthology.orchestra.run.smithi136.stdout:} 2021-01-15T19:11:46.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2021-01-15T19:11:46.217 INFO:tasks.ceph.mon.a.smithi001.stderr:2021-01-15T19:11:46.213+0000 7f9995812700 -1 mon.a@0(leader).osd e213 definitely_dead 0 2021-01-15T19:11:46.226 INFO:tasks.ceph.osd.7.smithi136.stderr:2021-01-15T19:11:46.225+0000 7f57959a0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34827) UID: 0 2021-01-15T19:11:46.300 INFO:teuthology.orchestra.run.smithi001.stderr:nodeep-scrub is set 2021-01-15T19:11:46.301 INFO:teuthology.orchestra.run.smithi001.stderr:marked out osd.0. 2021-01-15T19:11:46.316 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:46.316 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:46.317 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:46.317 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:46.327 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:46.325+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:46.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2021-01-15T19:11:46.428 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:46.427+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:46.499 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:46.499 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:46.500 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:46.500 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:46.500 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:46.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2021-01-15T19:11:46.553 INFO:tasks.ceph.osd.3.smithi001.stderr:2021-01-15T19:11:46.552+0000 7f83d806b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 34831) UID: 0 2021-01-15T19:11:46.630 INFO:tasks.ceph.osd.6.smithi136.stderr:2021-01-15T19:11:46.629+0000 7f3fac8ee700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 48091) UID: 0 2021-01-15T19:11:46.643 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:46.643 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:46.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:46.644 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:46.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.644 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:543 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:46.644 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:46.644 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.20545120400001, 2021-01-15T19:11:46.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002, 2021-01-15T19:11:46.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.645 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.646 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 543 2021-01-15T19:11:46.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.647 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.647 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.647 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.647 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:46.647 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.648 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.648 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435134+0000", 2021-01-15T19:11:46.649 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8381999999999999e-05 2021-01-15T19:11:46.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.649 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.649 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435182+0000", 2021-01-15T19:11:46.650 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.4590000000000004e-06 2021-01-15T19:11:46.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.650 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.650 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435192+0000", 2021-01-15T19:11:46.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8121000000000001e-05 2021-01-15T19:11:46.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435210+0000", 2021-01-15T19:11:46.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3079999999999997e-06 2021-01-15T19:11:46.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.652 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435219+0000", 2021-01-15T19:11:46.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6322999999999995e-05 2021-01-15T19:11:46.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435286+0000", 2021-01-15T19:11:46.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6500000000000002e-06 2021-01-15T19:11:46.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435289+0000", 2021-01-15T19:11:46.656 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.75863756000000004 2021-01-15T19:11:46.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.193927+0000", 2021-01-15T19:11:46.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001139284 2021-01-15T19:11:46.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195066+0000", 2021-01-15T19:11:46.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.064575009000000003 2021-01-15T19:11:46.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259641+0000", 2021-01-15T19:11:46.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000158028 2021-01-15T19:11:46.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.259799+0000", 2021-01-15T19:11:46.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00085663299999999998 2021-01-15T19:11:46.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.661 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.260656+0000", 2021-01-15T19:11:46.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0074796439999999997 2021-01-15T19:11:46.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268135+0000", 2021-01-15T19:11:46.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012027112 2021-01-15T19:11:46.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.664 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.664 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280162+0000", 2021-01-15T19:11:46.664 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032064799999999998 2021-01-15T19:11:46.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.665 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.665 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280483+0000", 2021-01-15T19:11:46.665 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84534916100000002 2021-01-15T19:11:46.665 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.666 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.666 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.666 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.667 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:544 3.14s1 3:2d79b238:::smithi00140323-40:head [rollback 19] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:46.667 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:46.667 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.20531190400001, 2021-01-15T19:11:46.667 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003, 2021-01-15T19:11:46.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.668 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 544 2021-01-15T19:11:46.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.669 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:46.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435273+0000", 2021-01-15T19:11:46.671 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1449000000000002e-05 2021-01-15T19:11:46.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435315+0000", 2021-01-15T19:11:46.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.0040000000000002e-06 2021-01-15T19:11:46.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.672 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435322+0000", 2021-01-15T19:11:46.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4369000000000001e-05 2021-01-15T19:11:46.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435336+0000", 2021-01-15T19:11:46.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1499999999999998e-06 2021-01-15T19:11:46.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.674 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435341+0000", 2021-01-15T19:11:46.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.5084000000000003e-05 2021-01-15T19:11:46.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435396+0000", 2021-01-15T19:11:46.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9139999999999998e-06 2021-01-15T19:11:46.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435398+0000", 2021-01-15T19:11:46.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76009075699999995 2021-01-15T19:11:46.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195489+0000", 2021-01-15T19:11:46.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00010094099999999999 2021-01-15T19:11:46.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.195590+0000", 2021-01-15T19:11:46.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.067371118999999993 2021-01-15T19:11:46.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.262961+0000", 2021-01-15T19:11:46.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015824099999999999 2021-01-15T19:11:46.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.681 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263119+0000", 2021-01-15T19:11:46.681 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073152500000000001 2021-01-15T19:11:46.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.681 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.682 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.263851+0000", 2021-01-15T19:11:46.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042879160000000001 2021-01-15T19:11:46.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.682 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.682 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.683 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268139+0000", 2021-01-15T19:11:46.683 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012519430999999999 2021-01-15T19:11:46.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280658+0000", 2021-01-15T19:11:46.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00030778199999999999 2021-01-15T19:11:46.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.684 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280966+0000", 2021-01-15T19:11:46.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84569268200000003 2021-01-15T19:11:46.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.685 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.686 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:545 3.14s1 3:2d79b238:::smithi00140323-40:head [append 0~0] snapc 1c=[1b,19,14] ondisk+write+known_if_redirected e55)", 2021-01-15T19:11:46.686 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:46.687 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.20519079799999, 2021-01-15T19:11:46.687 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996, 2021-01-15T19:11:46.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.687 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.688 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 545 2021-01-15T19:11:46.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.689 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.689 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.689 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.689 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:46.690 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.690 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.690 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.691 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435394+0000", 2021-01-15T19:11:46.691 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4792000000000001e-05 2021-01-15T19:11:46.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.691 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.692 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435439+0000", 2021-01-15T19:11:46.692 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7550000000000006e-06 2021-01-15T19:11:46.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.693 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.693 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435449+0000", 2021-01-15T19:11:46.693 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5778999999999999e-05 2021-01-15T19:11:46.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.694 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.694 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435465+0000", 2021-01-15T19:11:46.694 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.8439999999999997e-06 2021-01-15T19:11:46.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.695 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.695 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435470+0000", 2021-01-15T19:11:46.695 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9724000000000001e-05 2021-01-15T19:11:46.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.696 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.696 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435509+0000", 2021-01-15T19:11:46.696 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2979999999999999e-06 2021-01-15T19:11:46.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.697 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.697 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:43.435515+0000", 2021-01-15T19:11:46.698 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.76145251800000002 2021-01-15T19:11:46.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.698 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.698 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.196967+0000", 2021-01-15T19:11:46.699 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020326 2021-01-15T19:11:46.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.699 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.699 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.700 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.197170+0000", 2021-01-15T19:11:46.700 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.069537959999999996 2021-01-15T19:11:46.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.700 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.701 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266708+0000", 2021-01-15T19:11:46.701 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016873800000000001 2021-01-15T19:11:46.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.701 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.701 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.702 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.266877+0000", 2021-01-15T19:11:46.702 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010065670000000001 2021-01-15T19:11:46.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.703 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.703 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.267884+0000", 2021-01-15T19:11:46.703 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010185699999999999 2021-01-15T19:11:46.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.704 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.704 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268902+0000", 2021-01-15T19:11:46.704 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012230393000000001 2021-01-15T19:11:46.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.705 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.705 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281133+0000", 2021-01-15T19:11:46.705 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000293529 2021-01-15T19:11:46.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.706 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.706 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.706 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.281426+0000", 2021-01-15T19:11:46.706 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.84603172699999996 2021-01-15T19:11:46.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.707 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.708 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:554 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~532480] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:46.708 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:46.708 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 182.43490992400001, 2021-01-15T19:11:46.708 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571, 2021-01-15T19:11:46.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.709 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 554 2021-01-15T19:11:46.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.710 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.710 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.710 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:46.711 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.711 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.711 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205675+0000", 2021-01-15T19:11:46.712 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032006 2021-01-15T19:11:46.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.712 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.712 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.205995+0000", 2021-01-15T19:11:46.713 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.2820000000000002e-06 2021-01-15T19:11:46.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.713 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.713 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206005+0000", 2021-01-15T19:11:46.714 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0823999999999999e-05 2021-01-15T19:11:46.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.714 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.715 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206025+0000", 2021-01-15T19:11:46.715 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1130000000000001e-06 2021-01-15T19:11:46.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.715 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206034+0000", 2021-01-15T19:11:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7249e-05 2021-01-15T19:11:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.717 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206091+0000", 2021-01-15T19:11:46.717 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000805782 2021-01-15T19:11:46.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.717 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.718 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.206897+0000", 2021-01-15T19:11:46.718 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.088435151000000004 2021-01-15T19:11:46.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.719 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.719 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295332+0000", 2021-01-15T19:11:46.719 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013886500000000001 2021-01-15T19:11:46.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.719 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.720 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.720 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.295471+0000", 2021-01-15T19:11:46.720 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001063729 2021-01-15T19:11:46.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.720 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.721 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.721 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.296534+0000", 2021-01-15T19:11:46.721 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016490666000000001 2021-01-15T19:11:46.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.721 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.722 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.722 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313025+0000", 2021-01-15T19:11:46.722 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013230100000000001 2021-01-15T19:11:46.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.722 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.723 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.723 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313157+0000", 2021-01-15T19:11:46.723 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031054900000000002 2021-01-15T19:11:46.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.724 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.724 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313468+0000", 2021-01-15T19:11:46.724 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.107792571 2021-01-15T19:11:46.724 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.724 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.725 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.725 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:555 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~737280] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:46.726 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:46.726 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 182.425931746, 2021-01-15T19:11:46.726 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737, 2021-01-15T19:11:46.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.727 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 555 2021-01-15T19:11:46.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.728 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.728 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.728 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.728 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:46.728 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.729 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.729 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.729 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214653+0000", 2021-01-15T19:11:46.729 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044236299999999998 2021-01-15T19:11:46.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.732 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.733 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215096+0000", 2021-01-15T19:11:46.734 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6710000000000005e-06 2021-01-15T19:11:46.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.734 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.734 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215104+0000", 2021-01-15T19:11:46.735 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0553999999999999e-05 2021-01-15T19:11:46.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.735 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.735 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215115+0000", 2021-01-15T19:11:46.736 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:46.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.736 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.737 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.737 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215122+0000", 2021-01-15T19:11:46.737 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4806000000000003e-05 2021-01-15T19:11:46.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.738 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.738 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.738 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:46.738 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013694600000000001 2021-01-15T19:11:46.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.739 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.739 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.739 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215324+0000", 2021-01-15T19:11:46.739 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.087559974999999998 2021-01-15T19:11:46.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.740 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.740 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.740 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.302884+0000", 2021-01-15T19:11:46.740 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00016237599999999999 2021-01-15T19:11:46.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.741 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.741 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.303047+0000", 2021-01-15T19:11:46.741 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0011750129999999999 2021-01-15T19:11:46.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.742 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.742 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.304222+0000", 2021-01-15T19:11:46.742 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0096982860000000004 2021-01-15T19:11:46.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.743 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.743 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.313920+0000", 2021-01-15T19:11:46.743 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118921 2021-01-15T19:11:46.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.744 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.744 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.744 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.314039+0000", 2021-01-15T19:11:46.744 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0020293870000000001 2021-01-15T19:11:46.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.745 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.745 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.745 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316068+0000", 2021-01-15T19:11:46.745 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.101414737 2021-01-15T19:11:46.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.747 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:556 3.11s1 3:8f2f8304:::smithi00140323-26:head [append 0~466944] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:46.747 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:46.747 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 182.42541204599999, 2021-01-15T19:11:46.747 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999, 2021-01-15T19:11:46.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.748 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 556 2021-01-15T19:11:46.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.749 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.749 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.749 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.750 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:46.750 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.751 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.751 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215173+0000", 2021-01-15T19:11:46.751 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000321117 2021-01-15T19:11:46.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.751 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.752 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.752 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215494+0000", 2021-01-15T19:11:46.752 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0329999999999999e-06 2021-01-15T19:11:46.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.753 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.753 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.753 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215499+0000", 2021-01-15T19:11:46.753 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.6560000000000004e-06 2021-01-15T19:11:46.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.754 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.754 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.755 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215509+0000", 2021-01-15T19:11:46.755 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3210000000000004e-06 2021-01-15T19:11:46.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.756 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.756 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215514+0000", 2021-01-15T19:11:46.756 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001163068 2021-01-15T19:11:46.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.757 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.757 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216677+0000", 2021-01-15T19:11:46.757 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000120359 2021-01-15T19:11:46.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.758 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.758 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.758 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216798+0000", 2021-01-15T19:11:46.759 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.090147922000000005 2021-01-15T19:11:46.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.759 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.759 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.759 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.306946+0000", 2021-01-15T19:11:46.760 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000172335 2021-01-15T19:11:46.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.760 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.760 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.307118+0000", 2021-01-15T19:11:46.761 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0010305819999999999 2021-01-15T19:11:46.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.761 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.761 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.761 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.308149+0000", 2021-01-15T19:11:46.762 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0079247580000000005 2021-01-15T19:11:46.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.762 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.762 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.762 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316073+0000", 2021-01-15T19:11:46.763 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012558599999999999 2021-01-15T19:11:46.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.763 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.763 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316199+0000", 2021-01-15T19:11:46.764 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00032344900000000001 2021-01-15T19:11:46.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.764 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.764 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.764 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316522+0000", 2021-01-15T19:11:46.765 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.10134918599999999 2021-01-15T19:11:46.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.765 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.765 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.766 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:557 3.11s1 3:8f2f8304:::smithi00140323-26:head [setxattr _header (57) in=7b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:46.766 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:46.766 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 182.423731092, 2021-01-15T19:11:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134, 2021-01-15T19:11:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.767 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.768 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 557 2021-01-15T19:11:46.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.768 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.768 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.769 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:46.769 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.769 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.769 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.770 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216854+0000", 2021-01-15T19:11:46.770 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4469000000000003e-05 2021-01-15T19:11:46.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.770 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.771 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.771 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216909+0000", 2021-01-15T19:11:46.771 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2400000000000001e-06 2021-01-15T19:11:46.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.772 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.772 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216916+0000", 2021-01-15T19:11:46.772 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5065999999999999e-05 2021-01-15T19:11:46.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.772 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.773 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.773 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216931+0000", 2021-01-15T19:11:46.773 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0809999999999999e-06 2021-01-15T19:11:46.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.774 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.774 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216937+0000", 2021-01-15T19:11:46.774 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.9648999999999998e-05 2021-01-15T19:11:46.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.775 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.775 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:46.775 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012357700000000001 2021-01-15T19:11:46.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.775 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.776 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.776 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217110+0000", 2021-01-15T19:11:46.776 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.092455320999999993 2021-01-15T19:11:46.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.777 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.777 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309565+0000", 2021-01-15T19:11:46.777 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017028900000000001 2021-01-15T19:11:46.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.777 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.778 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.778 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.309736+0000", 2021-01-15T19:11:46.778 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00067988499999999997 2021-01-15T19:11:46.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.779 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.779 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.779 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310416+0000", 2021-01-15T19:11:46.779 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061112720000000001 2021-01-15T19:11:46.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316527+0000", 2021-01-15T19:11:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122631 2021-01-15T19:11:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.781 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.781 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.316650+0000", 2021-01-15T19:11:46.781 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078065399999999998 2021-01-15T19:11:46.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.782 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.782 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.782 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317430+0000", 2021-01-15T19:11:46.782 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.100576134 2021-01-15T19:11:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:558 3.11s1 3:8f2f8304:::smithi00140323-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 182.42359813799999, 2021-01-15T19:11:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772, 2021-01-15T19:11:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 558 2021-01-15T19:11:46.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.786 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.786 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.786 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.786 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:46.787 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.787 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.788 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.216987+0000", 2021-01-15T19:11:46.788 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4608999999999999e-05 2021-01-15T19:11:46.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.788 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.789 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217032+0000", 2021-01-15T19:11:46.789 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3050000000000004e-06 2021-01-15T19:11:46.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.789 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.790 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217039+0000", 2021-01-15T19:11:46.790 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3875e-05 2021-01-15T19:11:46.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.790 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217053+0000", 2021-01-15T19:11:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4650000000000002e-06 2021-01-15T19:11:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.791 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217058+0000", 2021-01-15T19:11:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6447999999999996e-05 2021-01-15T19:11:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.792 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217145+0000", 2021-01-15T19:11:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000108391 2021-01-15T19:11:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.793 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for missing object", 2021-01-15T19:11:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.217253+0000", 2021-01-15T19:11:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.093536899000000007 2021-01-15T19:11:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.794 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310790+0000", 2021-01-15T19:11:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013306400000000001 2021-01-15T19:11:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.795 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.310923+0000", 2021-01-15T19:11:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0065580309999999998 2021-01-15T19:11:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.796 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317481+0000", 2021-01-15T19:11:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015264599999999999 2021-01-15T19:11:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.317634+0000", 2021-01-15T19:11:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0042530390000000001 2021-01-15T19:11:46.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.321887+0000", 2021-01-15T19:11:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.104899772 2021-01-15T19:11:46.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.800 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:627 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 10] snapc 28=[28,26,25,24,22,21,20,1f,1b,19,14] ondisk+write+known_if_redirected e65)", 2021-01-15T19:11:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 173.46106686799999, 2021-01-15T19:11:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002, 2021-01-15T19:11:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 627 2021-01-15T19:11:46.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:46.803 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179518+0000", 2021-01-15T19:11:46.804 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4209999999999996e-05 2021-01-15T19:11:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179593+0000", 2021-01-15T19:11:46.805 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0390999999999999e-05 2021-01-15T19:11:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179603+0000", 2021-01-15T19:11:46.806 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7147999999999999e-05 2021-01-15T19:11:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.807 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179620+0000", 2021-01-15T19:11:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.7419999999999992e-06 2021-01-15T19:11:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.808 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179630+0000", 2021-01-15T19:11:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4660999999999995e-05 2021-01-15T19:11:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.809 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179704+0000", 2021-01-15T19:11:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2901999999999999e-05 2021-01-15T19:11:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.810 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "op must wait for map", 2021-01-15T19:11:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179717+0000", 2021-01-15T19:11:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0085334399999999998 2021-01-15T19:11:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.811 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188251+0000", 2021-01-15T19:11:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013033899999999999 2021-01-15T19:11:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188381+0000", 2021-01-15T19:11:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014791469999999999 2021-01-15T19:11:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.813 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.203173+0000", 2021-01-15T19:11:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0049592009999999999 2021-01-15T19:11:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.208132+0000", 2021-01-15T19:11:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0092268839999999994 2021-01-15T19:11:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.815 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217359+0000", 2021-01-15T19:11:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.9407000000000006e-05 2021-01-15T19:11:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.816 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.217448+0000", 2021-01-15T19:11:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037929795000000002 2021-01-15T19:11:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.817 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:628 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e65)", 2021-01-15T19:11:46.818 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 173.460900275, 2021-01-15T19:11:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003, 2021-01-15T19:11:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:46.819 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 628 2021-01-15T19:11:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.820 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179685+0000", 2021-01-15T19:11:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1811000000000003e-05 2021-01-15T19:11:46.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179727+0000", 2021-01-15T19:11:46.823 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.8250000000000005e-06 2021-01-15T19:11:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179735+0000", 2021-01-15T19:11:46.824 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4041e-05 2021-01-15T19:11:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.825 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179749+0000", 2021-01-15T19:11:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1659999999999999e-06 2021-01-15T19:11:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.826 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179755+0000", 2021-01-15T19:11:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6247e-05 2021-01-15T19:11:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.827 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179791+0000", 2021-01-15T19:11:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5590000000000006e-06 2021-01-15T19:11:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting_for_map not empty", 2021-01-15T19:11:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.179800+0000", 2021-01-15T19:11:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087864029999999999 2021-01-15T19:11:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.829 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188586+0000", 2021-01-15T19:11:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.9629000000000001e-05 2021-01-15T19:11:46.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.188646+0000", 2021-01-15T19:11:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.018842515000000001 2021-01-15T19:11:46.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207488+0000", 2021-01-15T19:11:46.832 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109032 2021-01-15T19:11:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.207597+0000", 2021-01-15T19:11:46.833 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011278214 2021-01-15T19:11:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.834 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218875+0000", 2021-01-15T19:11:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116032 2021-01-15T19:11:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.835 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.218991+0000", 2021-01-15T19:11:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000133657 2021-01-15T19:11:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:53.219125+0000", 2021-01-15T19:11:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.039440131000000003 2021-01-15T19:11:46.837 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.838 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:869 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~737280] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.262361481, 2021-01-15T19:11:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005, 2021-01-15T19:11:46.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.840 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 869 2021-01-15T19:11:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.841 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.843 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.843 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378224+0000", 2021-01-15T19:11:46.843 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045287600000000001 2021-01-15T19:11:46.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.844 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.844 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378677+0000", 2021-01-15T19:11:46.844 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2714999999999999e-05 2021-01-15T19:11:46.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378689+0000", 2021-01-15T19:11:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.0318000000000001e-05 2021-01-15T19:11:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378710+0000", 2021-01-15T19:11:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0125999999999999e-05 2021-01-15T19:11:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378720+0000", 2021-01-15T19:11:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.6268999999999999e-05 2021-01-15T19:11:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.847 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378776+0000", 2021-01-15T19:11:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.5860000000000001e-06 2021-01-15T19:11:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.848 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.378780+0000", 2021-01-15T19:11:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96191073900000001 2021-01-15T19:11:46.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340690+0000", 2021-01-15T19:11:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00023148099999999999 2021-01-15T19:11:46.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.340922+0000", 2021-01-15T19:11:46.851 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0017903789999999999 2021-01-15T19:11:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.342712+0000", 2021-01-15T19:11:46.852 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022109757000000001 2021-01-15T19:11:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364822+0000", 2021-01-15T19:11:46.853 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000401882 2021-01-15T19:11:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.854 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365224+0000", 2021-01-15T19:11:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00018631799999999999 2021-01-15T19:11:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.855 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365410+0000", 2021-01-15T19:11:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98718644600000005 2021-01-15T19:11:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.856 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:870 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~425984] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:46.857 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.26009143600001, 2021-01-15T19:11:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001, 2021-01-15T19:11:46.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 870 2021-01-15T19:11:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.860 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.861 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380494+0000", 2021-01-15T19:11:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027999900000000001 2021-01-15T19:11:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.862 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380774+0000", 2021-01-15T19:11:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.04e-05 2021-01-15T19:11:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.863 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380784+0000", 2021-01-15T19:11:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8039000000000001e-05 2021-01-15T19:11:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.864 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380802+0000", 2021-01-15T19:11:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.7229999999999993e-06 2021-01-15T19:11:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.865 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380811+0000", 2021-01-15T19:11:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.1190000000000002e-05 2021-01-15T19:11:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.866 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380872+0000", 2021-01-15T19:11:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.8200000000000001e-06 2021-01-15T19:11:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.867 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.380875+0000", 2021-01-15T19:11:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96657840100000003 2021-01-15T19:11:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.868 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347453+0000", 2021-01-15T19:11:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022753899999999999 2021-01-15T19:11:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.869 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.347681+0000", 2021-01-15T19:11:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001031045 2021-01-15T19:11:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.348712+0000", 2021-01-15T19:11:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016165039999999999 2021-01-15T19:11:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.871 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364877+0000", 2021-01-15T19:11:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00076613600000000005 2021-01-15T19:11:46.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365643+0000", 2021-01-15T19:11:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026645800000000002 2021-01-15T19:11:46.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.365910+0000", 2021-01-15T19:11:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98541579000000001 2021-01-15T19:11:46.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.875 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:871 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~761856] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.25555324499999, 2021-01-15T19:11:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002, 2021-01-15T19:11:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 871 2021-01-15T19:11:46.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.878 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.879 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385032+0000", 2021-01-15T19:11:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000181853 2021-01-15T19:11:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.880 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385214+0000", 2021-01-15T19:11:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2919999999999996e-06 2021-01-15T19:11:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.881 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385222+0000", 2021-01-15T19:11:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3509000000000001e-05 2021-01-15T19:11:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.882 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385236+0000", 2021-01-15T19:11:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4390000000000003e-06 2021-01-15T19:11:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385243+0000", 2021-01-15T19:11:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.4523999999999994e-05 2021-01-15T19:11:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385308+0000", 2021-01-15T19:11:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1609999999999996e-06 2021-01-15T19:11:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.385312+0000", 2021-01-15T19:11:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.96497541399999998 2021-01-15T19:11:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350287+0000", 2021-01-15T19:11:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000227748 2021-01-15T19:11:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.350515+0000", 2021-01-15T19:11:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.001192341 2021-01-15T19:11:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.351707+0000", 2021-01-15T19:11:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013170889 2021-01-15T19:11:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.364878+0000", 2021-01-15T19:11:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0012351969999999999 2021-01-15T19:11:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366113+0000", 2021-01-15T19:11:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00027276399999999999 2021-01-15T19:11:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366386+0000", 2021-01-15T19:11:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98135413100000002 2021-01-15T19:11:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:872 3.16s0 3:6feca5e1:::smithi00140323-9:head [append 0~417792] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.25296737299999, 2021-01-15T19:11:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197, 2021-01-15T19:11:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 872 2021-01-15T19:11:46.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:46.898 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387618+0000", 2021-01-15T19:11:46.899 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000287452 2021-01-15T19:11:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387905+0000", 2021-01-15T19:11:46.900 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0193999999999999e-05 2021-01-15T19:11:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387915+0000", 2021-01-15T19:11:46.901 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.8244999999999999e-05 2021-01-15T19:11:46.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.903 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387934+0000", 2021-01-15T19:11:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4569999999999999e-06 2021-01-15T19:11:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.904 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387941+0000", 2021-01-15T19:11:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6830999999999998e-05 2021-01-15T19:11:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388008+0000", 2021-01-15T19:11:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2080000000000002e-06 2021-01-15T19:11:46.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388012+0000", 2021-01-15T19:11:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.97892874399999996 2021-01-15T19:11:46.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.366941+0000", 2021-01-15T19:11:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022575000000000001 2021-01-15T19:11:46.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367167+0000", 2021-01-15T19:11:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00074294400000000005 2021-01-15T19:11:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.910 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.367910+0000", 2021-01-15T19:11:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0028177559999999998 2021-01-15T19:11:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370727+0000", 2021-01-15T19:11:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.003028658 2021-01-15T19:11:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.912 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.373756+0000", 2021-01-15T19:11:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031095800000000001 2021-01-15T19:11:46.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.374067+0000", 2021-01-15T19:11:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.986449197 2021-01-15T19:11:46.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.915 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:873 3.16s0 3:6feca5e1:::smithi00140323-9:head [setxattr _header (56) in=7b] snapc 3a=[3a,38,36,2f,2a,26,25,22,1b,19] ondisk+write+known_if_redirected e84)", 2021-01-15T19:11:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.25259303799999, 2021-01-15T19:11:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998, 2021-01-15T19:11:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 873 2021-01-15T19:11:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.918 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.919 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.387992+0000", 2021-01-15T19:11:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.0600000000000003e-05 2021-01-15T19:11:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.920 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388053+0000", 2021-01-15T19:11:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 9.3409999999999992e-06 2021-01-15T19:11:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388062+0000", 2021-01-15T19:11:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3380000000000001e-05 2021-01-15T19:11:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.922 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388075+0000", 2021-01-15T19:11:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.434e-06 2021-01-15T19:11:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388081+0000", 2021-01-15T19:11:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.2616e-05 2021-01-15T19:11:46.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388124+0000", 2021-01-15T19:11:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.559e-06 2021-01-15T19:11:46.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388125+0000", 2021-01-15T19:11:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98100309200000002 2021-01-15T19:11:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369128+0000", 2021-01-15T19:11:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00022151000000000001 2021-01-15T19:11:46.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.369350+0000", 2021-01-15T19:11:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00073772699999999998 2021-01-15T19:11:46.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370087+0000", 2021-01-15T19:11:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000641057 2021-01-15T19:11:46.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370728+0000", 2021-01-15T19:11:46.930 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0052814000000000003 2021-01-15T19:11:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.931 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376010+0000", 2021-01-15T19:11:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00025449400000000002 2021-01-15T19:11:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.932 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376264+0000", 2021-01-15T19:11:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98827220999999998 2021-01-15T19:11:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.933 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:874 3.16s0 3:6feca5e1:::smithi00140323-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected e84)", 2021-01-15T19:11:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:46.934 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 154.25245654299999, 2021-01-15T19:11:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001, 2021-01-15T19:11:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:46.935 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 874 2021-01-15T19:11:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.936 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388129+0000", 2021-01-15T19:11:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8787e-05 2021-01-15T19:11:46.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388167+0000", 2021-01-15T19:11:46.939 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7530000000000004e-06 2021-01-15T19:11:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388174+0000", 2021-01-15T19:11:46.940 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3091000000000001e-05 2021-01-15T19:11:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388187+0000", 2021-01-15T19:11:46.941 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.1780000000000002e-06 2021-01-15T19:11:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.942 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388192+0000", 2021-01-15T19:11:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5238999999999998e-05 2021-01-15T19:11:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.943 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388238+0000", 2021-01-15T19:11:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.9209999999999999e-06 2021-01-15T19:11:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:46.944 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:12.388240+0000", 2021-01-15T19:11:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.98216671300000002 2021-01-15T19:11:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.945 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370406+0000", 2021-01-15T19:11:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00020334899999999999 2021-01-15T19:11:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:46.946 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.370610+0000", 2021-01-15T19:11:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.006273107 2021-01-15T19:11:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.947 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.376883+0000", 2021-01-15T19:11:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00021221799999999999 2021-01-15T19:11:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.948 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.377095+0000", 2021-01-15T19:11:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0027049449999999998 2021-01-15T19:11:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.949 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:13.379800+0000", 2021-01-15T19:11:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.99167130100000001 2021-01-15T19:11:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:890 3.5s0 3:a5ddc9cd:::smithi00140323-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e86)", 2021-01-15T19:11:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:46.951 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 152.270058823, 2021-01-15T19:11:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001, 2021-01-15T19:11:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.952 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 890 2021-01-15T19:11:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370526+0000", 2021-01-15T19:11:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.6681000000000003e-05 2021-01-15T19:11:46.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370563+0000", 2021-01-15T19:11:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.1359999999999999e-06 2021-01-15T19:11:46.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370570+0000", 2021-01-15T19:11:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.1472e-05 2021-01-15T19:11:46.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370582+0000", 2021-01-15T19:11:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.968e-06 2021-01-15T19:11:46.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.370586+0000", 2021-01-15T19:11:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00044656600000000002 2021-01-15T19:11:46.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371032+0000", 2021-01-15T19:11:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5026999999999998e-05 2021-01-15T19:11:46.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.371117+0000", 2021-01-15T19:11:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.030129165999999999 2021-01-15T19:11:46.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401246+0000", 2021-01-15T19:11:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015901899999999999 2021-01-15T19:11:46.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.401405+0000", 2021-01-15T19:11:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0033779080000000002 2021-01-15T19:11:46.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404783+0000", 2021-01-15T19:11:46.964 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.0012999999999994e-05 2021-01-15T19:11:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404863+0000", 2021-01-15T19:11:46.965 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.6349999999999998e-05 2021-01-15T19:11:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:09:14.404950+0000", 2021-01-15T19:11:46.966 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.034423306000000001 2021-01-15T19:11:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1488 3.5s0 3:a12cbe63:::smithi00140323-21:head [copy-from ver 205] snapc 7b=[7b,7a,79,78,73,72,70,6f,6e,6b,67,66,64,63,60,56,4b] ondisk+write+known_if_redirected e153)", 2021-01-15T19:11:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 85.123305764999998, 2021-01-15T19:11:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999, 2021-01-15T19:11:46.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1488 2021-01-15T19:11:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.970 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.971 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517279+0000", 2021-01-15T19:11:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6997999999999998e-05 2021-01-15T19:11:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.972 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517346+0000", 2021-01-15T19:11:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2140999999999999e-05 2021-01-15T19:11:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.973 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517359+0000", 2021-01-15T19:11:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.2308999999999999e-05 2021-01-15T19:11:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.974 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517381+0000", 2021-01-15T19:11:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0057999999999999e-05 2021-01-15T19:11:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.975 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517391+0000", 2021-01-15T19:11:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00017052199999999999 2021-01-15T19:11:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.976 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517561+0000", 2021-01-15T19:11:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015747800000000001 2021-01-15T19:11:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.977 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.978 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517719+0000", 2021-01-15T19:11:46.978 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.022669049 2021-01-15T19:11:46.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.978 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:46.979 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.540388+0000", 2021-01-15T19:11:46.979 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.013707555999999999 2021-01-15T19:11:46.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.979 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.979 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:46.980 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554096+0000", 2021-01-15T19:11:46.980 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00082751799999999996 2021-01-15T19:11:46.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.980 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.980 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.554923+0000", 2021-01-15T19:11:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152168 2021-01-15T19:11:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.981 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.555075+0000", 2021-01-15T19:11:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.037795796999999999 2021-01-15T19:11:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:1489 3.5s0 3:a12cbe63:::smithi00140323-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e153)", 2021-01-15T19:11:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:46.983 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 85.123142833000003, 2021-01-15T19:11:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002, 2021-01-15T19:11:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:46.984 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 1489 2021-01-15T19:11:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:46.985 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:46.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517442+0000", 2021-01-15T19:11:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.1896e-05 2021-01-15T19:11:46.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517484+0000", 2021-01-15T19:11:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3869999999999997e-06 2021-01-15T19:11:46.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517492+0000", 2021-01-15T19:11:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6081000000000001e-05 2021-01-15T19:11:46.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517508+0000", 2021-01-15T19:11:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4539999999999998e-06 2021-01-15T19:11:46.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517513+0000", 2021-01-15T19:11:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000455265 2021-01-15T19:11:46.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.517968+0000", 2021-01-15T19:11:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5446999999999998e-05 2021-01-15T19:11:46.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.518044+0000", 2021-01-15T19:11:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.035520836 2021-01-15T19:11:46.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553565+0000", 2021-01-15T19:11:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013480299999999999 2021-01-15T19:11:46.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.553700+0000", 2021-01-15T19:11:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0044952000000000004 2021-01-15T19:11:46.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558195+0000", 2021-01-15T19:11:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.7904000000000006e-05 2021-01-15T19:11:46.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558263+0000", 2021-01-15T19:11:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2422999999999993e-05 2021-01-15T19:11:46.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:21.558345+0000", 2021-01-15T19:11:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.040902696000000002 2021-01-15T19:11:46.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:46.999 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDScrubReserve(3.11s1 RELEASE e159)", 2021-01-15T19:11:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 79.349470195999999, 2021-01-15T19:11:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998, 2021-01-15T19:11:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:47.001 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291115+0000", 2021-01-15T19:11:47.002 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6618e-05 2021-01-15T19:11:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291162+0000", 2021-01-15T19:11:47.003 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.1859999999999992e-06 2021-01-15T19:11:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291170+0000", 2021-01-15T19:11:47.004 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2869e-05 2021-01-15T19:11:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291183+0000", 2021-01-15T19:11:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6329999999999999e-06 2021-01-15T19:11:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291189+0000", 2021-01-15T19:11:47.006 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000544489 2021-01-15T19:11:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291734+0000", 2021-01-15T19:11:47.007 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.2059999999999999e-06 2021-01-15T19:11:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for peered", 2021-01-15T19:11:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:27.291737+0000", 2021-01-15T19:11:47.008 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.913936525 2021-01-15T19:11:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205673+0000", 2021-01-15T19:11:47.009 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.1534000000000001e-05 2021-01-15T19:11:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205695+0000", 2021-01-15T19:11:47.010 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.0520000000000002e-06 2021-01-15T19:11:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:10:28.205698+0000", 2021-01-15T19:11:47.011 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.91458311199999998 2021-01-15T19:11:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.013 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:47.014 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:47.013+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:47.014 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2021-01-15T19:11:47.115 INFO:tasks.ceph.osd.2.smithi001.stderr:2021-01-15T19:11:47.114+0000 7ff43f32f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 34826) UID: 0 2021-01-15T19:11:47.190 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:47.191 INFO:teuthology.orchestra.run.smithi001.stdout: "num_ops": 0 2021-01-15T19:11:47.191 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:47.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2021-01-15T19:11:47.215 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:47.215+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:47.317 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:47.316+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:47.379 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [], 2021-01-15T19:11:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "complaint_time": 30, 2021-01-15T19:11:47.379 INFO:teuthology.orchestra.run.smithi001.stdout: "num_blocked_ops": 0 2021-01-15T19:11:47.380 INFO:teuthology.orchestra.run.smithi001.stdout:} 2021-01-15T19:11:47.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2021-01-15T19:11:47.418 INFO:tasks.ceph.osd.4.smithi136.stderr:2021-01-15T19:11:47.416+0000 7f0cd4cb7700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34819) UID: 0 2021-01-15T19:11:47.519 INFO:tasks.ceph.osd.5.smithi136.stderr:2021-01-15T19:11:47.518+0000 7f0f68b60700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34822) UID: 0 2021-01-15T19:11:47.522 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2021-01-15T19:11:47.522 INFO:teuthology.orchestra.run.smithi001.stdout: "size": 20, 2021-01-15T19:11:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 600, 2021-01-15T19:11:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "ops": [ 2021-01-15T19:11:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:432 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [copy-from ver 4] snapc 1b=[1b,19,14,10] ondisk+write+known_if_redirected e51)", 2021-01-15T19:11:47.523 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 188.271222722, 2021-01-15T19:11:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001, 2021-01-15T19:11:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:47.524 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 432 2021-01-15T19:11:47.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.526 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.527 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248893+0000", 2021-01-15T19:11:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5273999999999999e-05 2021-01-15T19:11:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.528 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248958+0000", 2021-01-15T19:11:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.7312e-05 2021-01-15T19:11:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.529 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248976+0000", 2021-01-15T19:11:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 2.3631000000000001e-05 2021-01-15T19:11:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.530 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.248999+0000", 2021-01-15T19:11:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0998e-05 2021-01-15T19:11:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.531 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249010+0000", 2021-01-15T19:11:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012149199999999999 2021-01-15T19:11:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.532 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249132+0000", 2021-01-15T19:11:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000152604 2021-01-15T19:11:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249284+0000", 2021-01-15T19:11:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012527674000000001 2021-01-15T19:11:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.261812+0000", 2021-01-15T19:11:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0050175860000000001 2021-01-15T19:11:47.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.266830+0000", 2021-01-15T19:11:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0087921010000000001 2021-01-15T19:11:47.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275622+0000", 2021-01-15T19:11:47.537 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011765900000000001 2021-01-15T19:11:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.275739+0000", 2021-01-15T19:11:47.538 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.026846331000000001 2021-01-15T19:11:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.539 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:433 3.ds0 3:bd2d5ff7:::smithi00140323-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e51)", 2021-01-15T19:11:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 188.27103177800001, 2021-01-15T19:11:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002, 2021-01-15T19:11:47.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 433 2021-01-15T19:11:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.542 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.543 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249084+0000", 2021-01-15T19:11:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.4291e-05 2021-01-15T19:11:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.544 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249128+0000", 2021-01-15T19:11:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.5610000000000001e-06 2021-01-15T19:11:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.545 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249136+0000", 2021-01-15T19:11:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3901e-05 2021-01-15T19:11:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249150+0000", 2021-01-15T19:11:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8300000000000001e-06 2021-01-15T19:11:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.547 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249156+0000", 2021-01-15T19:11:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00037192399999999999 2021-01-15T19:11:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.548 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249527+0000", 2021-01-15T19:11:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.3480999999999997e-05 2021-01-15T19:11:47.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.550 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:47.551 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.249591+0000", 2021-01-15T19:11:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.015596918 2021-01-15T19:11:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.552 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265188+0000", 2021-01-15T19:11:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00011772799999999999 2021-01-15T19:11:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.553 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.265306+0000", 2021-01-15T19:11:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.010872550999999999 2021-01-15T19:11:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276178+0000", 2021-01-15T19:11:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000111939 2021-01-15T19:11:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276290+0000", 2021-01-15T19:11:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000124506 2021-01-15T19:11:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:39.276415+0000", 2021-01-15T19:11:47.557 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.027330630000000002 2021-01-15T19:11:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:550 3.ds0 3:b4eba64c:::smithi00140323-5:head [copy-from ver 8] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.33207968100001, 2021-01-15T19:11:47.559 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247, 2021-01-15T19:11:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.560 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 550 2021-01-15T19:11:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.561 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188036+0000", 2021-01-15T19:11:47.563 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.7757000000000003e-05 2021-01-15T19:11:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.564 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188094+0000", 2021-01-15T19:11:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.5059999999999995e-06 2021-01-15T19:11:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.565 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188102+0000", 2021-01-15T19:11:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5877999999999999e-05 2021-01-15T19:11:47.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188118+0000", 2021-01-15T19:11:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.0175e-05 2021-01-15T19:11:47.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.188128+0000", 2021-01-15T19:11:47.568 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0025502929999999999 2021-01-15T19:11:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190679+0000", 2021-01-15T19:11:47.569 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000122872 2021-01-15T19:11:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.570 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190802+0000", 2021-01-15T19:11:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.012275252 2021-01-15T19:11:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.571 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.203077+0000", 2021-01-15T19:11:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.016372111000000002 2021-01-15T19:11:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.219449+0000", 2021-01-15T19:11:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0069004219999999998 2021-01-15T19:11:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.573 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226349+0000", 2021-01-15T19:11:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00012298100000000001 2021-01-15T19:11:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.226472+0000", 2021-01-15T19:11:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.038436247 2021-01-15T19:11:47.575 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.576 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:551 3.ds0 3:b4eba64c:::smithi00140323-5:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.32935921999999, 2021-01-15T19:11:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999, 2021-01-15T19:11:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 551 2021-01-15T19:11:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.579 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.580 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190757+0000", 2021-01-15T19:11:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.0487e-05 2021-01-15T19:11:47.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190797+0000", 2021-01-15T19:11:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.3339999999999996e-06 2021-01-15T19:11:47.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190804+0000", 2021-01-15T19:11:47.583 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.4653e-05 2021-01-15T19:11:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190819+0000", 2021-01-15T19:11:47.584 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2439999999999996e-06 2021-01-15T19:11:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190826+0000", 2021-01-15T19:11:47.585 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000126372 2021-01-15T19:11:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.586 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190953+0000", 2021-01-15T19:11:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.0618999999999998e-05 2021-01-15T19:11:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.587 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191003+0000", 2021-01-15T19:11:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023677569999999998 2021-01-15T19:11:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.588 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214681+0000", 2021-01-15T19:11:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000116638 2021-01-15T19:11:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.589 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214797+0000", 2021-01-15T19:11:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.014116899 2021-01-15T19:11:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.590 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.228914+0000", 2021-01-15T19:11:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0048423260000000001 2021-01-15T19:11:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.591 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.233757+0000", 2021-01-15T19:11:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00094778199999999999 2021-01-15T19:11:47.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.234704+0000", 2021-01-15T19:11:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.043947923999999999 2021-01-15T19:11:47.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:552 3.as0 3:5dab47d9:::smithi00140323-29:head [copy-from ver 23] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.329246641, 2021-01-15T19:11:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006, 2021-01-15T19:11:47.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.596 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 552 2021-01-15T19:11:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.597 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190869+0000", 2021-01-15T19:11:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.4289000000000001e-05 2021-01-15T19:11:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.599 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190923+0000", 2021-01-15T19:11:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.9469999999999997e-06 2021-01-15T19:11:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.600 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190930+0000", 2021-01-15T19:11:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3114e-05 2021-01-15T19:11:47.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190943+0000", 2021-01-15T19:11:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.6200000000000001e-06 2021-01-15T19:11:47.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190950+0000", 2021-01-15T19:11:47.603 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026931900000000001 2021-01-15T19:11:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191219+0000", 2021-01-15T19:11:47.604 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000118785 2021-01-15T19:11:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191338+0000", 2021-01-15T19:11:47.605 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.056003024999999998 2021-01-15T19:11:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.606 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.247341+0000", 2021-01-15T19:11:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.032856913000000001 2021-01-15T19:11:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280198+0000", 2021-01-15T19:11:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00026656599999999998 2021-01-15T19:11:47.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280465+0000", 2021-01-15T19:11:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00013666000000000001 2021-01-15T19:11:47.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280601+0000", 2021-01-15T19:11:47.610 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.089732238000000006 2021-01-15T19:11:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.611 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:553 3.as0 3:5dab47d9:::smithi00140323-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected e56)", 2021-01-15T19:11:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.32911673699999, 2021-01-15T19:11:47.612 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002, 2021-01-15T19:11:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "started", 2021-01-15T19:11:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.613 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 553 2021-01-15T19:11:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.614 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.615 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.190999+0000", 2021-01-15T19:11:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.5153e-05 2021-01-15T19:11:47.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191044+0000", 2021-01-15T19:11:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.2320000000000004e-06 2021-01-15T19:11:47.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191051+0000", 2021-01-15T19:11:47.618 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.3159000000000001e-05 2021-01-15T19:11:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191065+0000", 2021-01-15T19:11:47.619 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.3619999999999997e-06 2021-01-15T19:11:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.620 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191070+0000", 2021-01-15T19:11:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00045204499999999999 2021-01-15T19:11:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.621 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191522+0000", 2021-01-15T19:11:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.8319000000000002e-05 2021-01-15T19:11:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.622 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for blocked object", 2021-01-15T19:11:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.191580+0000", 2021-01-15T19:11:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.077348005999999997 2021-01-15T19:11:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.623 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.268928+0000", 2021-01-15T19:11:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000115841 2021-01-15T19:11:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.624 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "waiting for rw locks", 2021-01-15T19:11:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269044+0000", 2021-01-15T19:11:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.017964720999999999 2021-01-15T19:11:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287009+0000", 2021-01-15T19:11:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000114177 2021-01-15T19:11:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.626 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287123+0000", 2021-01-15T19:11:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00015261499999999999 2021-01-15T19:11:47.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.287276+0000", 2021-01-15T19:11:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.096276630000000002 2021-01-15T19:11:47.628 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.629 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:564 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.30591146800001, 2021-01-15T19:11:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002, 2021-01-15T19:11:47.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.631 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 564 2021-01-15T19:11:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214204+0000", 2021-01-15T19:11:47.634 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9659000000000001e-05 2021-01-15T19:11:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.635 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214244+0000", 2021-01-15T19:11:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.3170000000000002e-06 2021-01-15T19:11:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.636 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214252+0000", 2021-01-15T19:11:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.5142e-05 2021-01-15T19:11:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214267+0000", 2021-01-15T19:11:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 6.5719999999999999e-06 2021-01-15T19:11:47.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214274+0000", 2021-01-15T19:11:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00077249600000000001 2021-01-15T19:11:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.640 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215046+0000", 2021-01-15T19:11:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014087999999999999 2021-01-15T19:11:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.641 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215187+0000", 2021-01-15T19:11:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00078774900000000002 2021-01-15T19:11:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.642 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.215975+0000", 2021-01-15T19:11:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.053708537000000001 2021-01-15T19:11:47.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269684+0000", 2021-01-15T19:11:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000105561 2021-01-15T19:11:47.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269789+0000", 2021-01-15T19:11:47.645 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000109189 2021-01-15T19:11:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269898+0000", 2021-01-15T19:11:47.646 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055694102000000002 2021-01-15T19:11:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:565 3.as0 3:59c0717b:::smithi00140323-48:head [rollback 1b] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.30579327000001, 2021-01-15T19:11:47.648 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998, 2021-01-15T19:11:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.649 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 565 2021-01-15T19:11:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.651 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214322+0000", 2021-01-15T19:11:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.8034999999999998e-05 2021-01-15T19:11:47.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214361+0000", 2021-01-15T19:11:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6120000000000001e-06 2021-01-15T19:11:47.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214368+0000", 2021-01-15T19:11:47.654 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.6631999999999999e-05 2021-01-15T19:11:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214385+0000", 2021-01-15T19:11:47.655 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.057e-06 2021-01-15T19:11:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.656 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214390+0000", 2021-01-15T19:11:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0090677259999999999 2021-01-15T19:11:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.657 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223458+0000", 2021-01-15T19:11:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00014284399999999999 2021-01-15T19:11:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.658 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.223600+0000", 2021-01-15T19:11:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00062283599999999998 2021-01-15T19:11:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.659 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.224223+0000", 2021-01-15T19:11:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.045725804000000002 2021-01-15T19:11:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.660 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269949+0000", 2021-01-15T19:11:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.4117999999999997e-05 2021-01-15T19:11:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.661 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270033+0000", 2021-01-15T19:11:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 8.2291000000000002e-05 2021-01-15T19:11:47.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.270115+0000", 2021-01-15T19:11:47.663 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.055792954999999998 2021-01-15T19:11:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "osd_op(client.4418.0:566 3.as0 3:59c0717b:::smithi00140323-48:head [append 0~0] snapc 1f=[1f,1b,19,14] ondisk+write+known_if_redirected e56)", 2021-01-15T19:11:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:47.665 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.30568058, 2021-01-15T19:11:47.666 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994, 2021-01-15T19:11:47.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "commit sent; apply or cleanup", 2021-01-15T19:11:47.666 INFO:teuthology.orchestra.run.smithi001.stdout: "client_info": { 2021-01-15T19:11:47.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client": "client.4418", 2021-01-15T19:11:47.667 INFO:teuthology.orchestra.run.smithi001.stdout: "client_addr": "172.21.15.1:0/3955300889", 2021-01-15T19:11:47.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tid": 566 2021-01-15T19:11:47.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.667 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214435+0000", 2021-01-15T19:11:47.669 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 3.9301000000000002e-05 2021-01-15T19:11:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "header_read", 2021-01-15T19:11:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214474+0000", 2021-01-15T19:11:47.670 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.6290000000000001e-06 2021-01-15T19:11:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "all_read", 2021-01-15T19:11:47.671 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214482+0000", 2021-01-15T19:11:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 1.2311e-05 2021-01-15T19:11:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.672 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "dispatched", 2021-01-15T19:11:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214494+0000", 2021-01-15T19:11:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 4.6639999999999997e-06 2021-01-15T19:11:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.673 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "queued_for_pg", 2021-01-15T19:11:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.214499+0000", 2021-01-15T19:11:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.054583101000000002 2021-01-15T19:11:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "reached_pg", 2021-01-15T19:11:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269082+0000", 2021-01-15T19:11:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 7.4751000000000003e-05 2021-01-15T19:11:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "started", 2021-01-15T19:11:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269157+0000", 2021-01-15T19:11:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.00031263200000000002 2021-01-15T19:11:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_started", 2021-01-15T19:11:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.269470+0000", 2021-01-15T19:11:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.011137397 2021-01-15T19:11:47.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "sub_op_committed", 2021-01-15T19:11:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280607+0000", 2021-01-15T19:11:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.000241473 2021-01-15T19:11:47.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "commit_sent", 2021-01-15T19:11:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.280848+0000", 2021-01-15T19:11:47.679 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.0061189089999999996 2021-01-15T19:11:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "done", 2021-01-15T19:11:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.286967+0000", 2021-01-15T19:11:47.680 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.072532167999999994 2021-01-15T19:11:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2021-01-15T19:11:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2021-01-15T19:11:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "description": "MOSDECSubOpWriteReply(3.as0 56/21 ECSubWriteReply(tid=171, last_complete=56'49, committed=1, applied=1))", 2021-01-15T19:11:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "initiated_at": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 183.30007882999999, 2021-01-15T19:11:47.682 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0.023475422999999999, 2021-01-15T19:11:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type_data": { 2021-01-15T19:11:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flag_point": "reached pg", 2021-01-15T19:11:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "events": [ 2021-01-15T19:11:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.683 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "initiated", 2021-01-15T19:11:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 0 2021-01-15T19:11:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.684 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "event": "throttled", 2021-01-15T19:11:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "time": "2021-01-15T19:08:44.220037+0000", 2021-01-15T19:11:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: "duration": 5.2114000000000003e-05 2021-01-15T19:11:47.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2021-01-15T19:11:47.686 INFO:teuthology.orchestra.run.smithi001.stdout: { 2021-01-15T19:11:47.686 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2021-01-15T19:11:47.686 DEBUG:teuthology.task.console_log:Killing console logger for smithi001 2021-01-15T19:11:47.687 DEBUG:teuthology.task.console_log:Killing console logger for smithi136 2021-01-15T19:11:47.688 DEBUG:teuthology.task.console_log:Killing console logger for smithi001 2021-01-15T19:11:47.688 DEBUG:teuthology.task.console_log:Killing console logger for smithi136 2021-01-15T19:11:47.689 DEBUG:teuthology.exit:Finished running handlers